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ABSTRACT 


The Navy’s various weapons —Ppre7ect officers 
reoupanely decide whether to use existing weapons 
technology or to extend into as yet undeveloped 
technology. For state-of-the-art (SOA) extensions, 


Mmsetal eCstimates ~oOL™ development cost frequently are 
inaccurate. This study first examines’ the background 
of methods utilized for SOA extension measurement. 
This study also reviews the cost estimating methods 
used by Litton Applied Technology, Inc. to estimate the 
development costs of the AN/ALR-67 Radar Warning 
Receiver, a specific SOA extension project. The 
principal findings are that regression analysis and 
geometric surface analysis are used to quantify SOA 
extensions, but only in theoretical applications. 
Litton Applied Technology uses the bottoms-up approach 
to estimate development costs. Tiemeemiture trends in 


defense cost estimating are also forecasted. 


lel. 


TABLE OF CONTENTS 


INTRODUCTION 


QQ 3 FS oem 


PURPOSE. 

BACKGROUND 

OBJECTIVE. : 

THE RESEARCH QUESTION. 

SCOPE AND LIMITATIONS. Se 
LITERATURE REVIEW AND METHODOLOGY. 
ORGANIZATION OF STUDY. 


LITERATURE REVIEW. 0 eee 


A. 
Be 


INTRODUCTION sonore. Gene 
OSTWALD’S BASIC TOOLS FOR COST 
ESTIMATING < © Flee oe el See 
cas Cost Estimating Relationships. 
2. Technological Forecasting. 

a. Cost Indices ; 
DODSON’S APPROACH TO QUANTITATIVE 
MEASUREMENT OF ADVANCES IN STATE- 
OF-THE-ART (JANUARY 1969). 
DODSON’S STUDY OF RESOURCE 
ESTIMATION IN DEVELOPMENT PROJECTS 
MEASURING TECHNOLOGICAL CHANGE IN 
AIRCRAFT TURBINE ENGINES 
HOVANSSIAN’S DESCRIPTION OF KEY 
PARAMETERS a 
DODSON’S DEVELOPMENT OF COST 
EQUATIONS TO MEASURE TECHNOLOGICAL 
CHANGE mee a. Pe ees 
GORDON AND MUNSON’S PROPOSED 
CONVENTION FOR MEASURING THE SOA 
OF PRODUCTS. 


10 
at 
2 
13 
15 


Abi 
1 


18 


18 


eo 


LY 


20 


aa 


24 


26 


28 


31 


irl. 


i V-. 


D Dis : = 1 = 


ITA 


I. ALEXANDER AND MITCHELL’S MEASUREMENT 
OF TECHNOLOGICAL CHANGE. ee fe 

J. MARTINO’S MEASUREMENT OF TECHNOLOGY 
USING TRADEOFF SURFACES. : 

K. DODSON’S MEASUREMENT OF SOA AND 
TECHNOLOGICAL ADVANCE. : 

L. MODELING RESEARCH AND DEVELOPMENT 
RESOURCE ALLOCATION. ee 

M. MANAGEMENT CONTROL OF NEW PRODUCT 
DEVELOPMENT PROJECTS Pe 

N. SUMMARIZATION OF RULES FOR BETTER 
ESTIMATES. “ae. eC 

O. EVALUATION OF KEY SOFTWARE 
ESTIMATING MODELS. 

P. SUMMARY. 


DEVELOPMENT OF COST ESTIMATING MODELS. 
INTRODUCTION. ere cs 

GROUND COMBAT SURVEILLANCE. : 
HIGH-ENERGY LASER SYSTEMS COST MODEL. 
UNMANNED SPACECRAFT COST MODEL. 
FREIMAN ANALYSIS OF SYSTEMS TECHNIQUE 
SUMMARY. 


mqtmid oO wm YS 


DESCRIPTION OF LITTON APPLIED TECHNOLOGY’S 


COST ESTIMATING SYSTEM. 
A. INTRODUCTION. ; 
B. BACKGROUND OF RADAR WARNING RECEIVERS 
1. General Background. 
Cou lSroryeot Radar Warning Receivers. 


C. PHYSICAL AND PERFORMANCE CHARACTERISTICS 
D. @iITON APPLIED TECHNOLOGY’S COST 
ESTIMATING SYSTEM. 


A LIBRARY 


+R somnor, 


33 


34 


36 


of 


38 


40 


41 
42 


43 
43 
43 
4‘7 
51 
D5 
09 


64 
64 
64 
64 
65 
67 


71 


COST ESTIMATION METHODOLOGY FOR 


DERIVATION OF DIRECT COSTS 3. . 3). Sues 
F. ACTUAL COSTS FOR THE AN/ALR-67 
RADAR WARNING RECEIVER ...... «© © (3a 
G. PREVIEW OF NEXT CHAPTER. .« .. =. « « 4 ee 
CASE ANALYSIS AND FUTURE TRENDS. .. =: . J ieee 
A. INTRODUCTION . 2.92 © «949s 6 3 6 6S) ee 
B. VARIANCE ANALYSIS OF THE DEVELOPMENT 
PROGRAM OF THE AN/ALR-67 RADAR 
WARNING RECEIVER... . 7 8 8 we 
C. OVERALL ANALYSIS OF LITTON APPLIED 
TECHNOLOGY’S COST ESTIMATING SYSTEM. .. 89 
1. Environmental. . . . 394 ; « « « ee 
2. Téechnical. “oo... eee ec cee 7 eee 
3. | SEPWUGCEUIC Wy cake wie Vo ec ees 
4. Psychosocial .. . *s « « « « « « se 
5. Managerial... ... «6% « = & < -eae 
6. Goals and Values ...+5:. .«. « « « « =e 
ls Control Mechanisms . «65 4 # <= .) <i 
D. CURRENT CONCERNS AND TRENDS IN COST 
ESTIMATING —.) eee. - 2 « re 
E. EMERGING ROLES COST ESTIMATORS MUST 
ASSUME IN THE FUTURE .~ 29 3) eee 
F. COMPARISON OF LITTON’S COST 


ESTIMATING PROGRAM TO CURRENT 


DIRECTION OF THE ESTIMATING FIELD. . . Jaee 
1. SOA Advance Measurements ..... . 97 
2.  EXPe@Prt. SVS Gems pcan eee int ercne oo ae 
3. Inthimence of “Costs er we eve ono 

Sta@e. . + ls Gat «eee ean se 
4. Understanding of Total Implications 

of New TeehnolG@ey: =. 9 . . . . a0 
Bh. Isolation of Cost Drivers... =]. . 300 


G. SUMMARY. 

Wel. SUMMARY AND CONCLUSIONS. 
A. INTRODUCTION 
Deeb ERATURE REVIEW. 
Cc. “COD MODELS: ee es 
DD. ERPGON’S COST ESTIMATING SYSTEM. 
Ee  CURRERMA DIRECTIONS 
Eo J reteruRE ROLES 
G. CONCLUSIONS. 5 ae 
H. RECOMMENDATIONS FOR FUTURE STUDY 


feote Or REFERENCES 
PerPAL DISTRIBUTION LIST. 


OG 


-102 
eS 
lO? 
~L0'3 
-104 
-105 
~-105 
.106 
- 107 
0S 
ele 


I. INTRODUCTION 


eee PURE OSs 

This thesis will describe how Litton Applied 
Technology of San Jose, California measured and 
estimated the cost of the state-of-the-art development 
of the AN/ALR-67 Radar Warning System. This case study 
will be utilized by Professor Willis Greer to test an 
improved cost estimating model which will relate 


measurements of state-of-the-art extensions with costs. 


B. BACKGROUND 

During the past three years, budget restraints have 
forced the Navy to tighten controls over expenditUree 
Por, “Major weapons systems. One area which has 
generated significant unplanned cost growth during the 
last two decades has been the extension of states-of- 
the-art (SOA). The cost impact of SOA extensions is 
most keenly felt during the demonstration and 
validation phase, and the full scale development phase 
of the acquisition process. During an interview with 
GTE cost analyst Stan Swales on 17 September 1987, he 
estimated most cost models for initial SOA development 
efforts were only accurate to within twenty to forty 
Petwec nee However, once the generic cost models were 
"calibrated" with actual cost parameters and the 
technical characteristics of the completed system, the 
range Of accuracy could be reduced to five to ten 
percent. Ren. 1 

During the concept exploration process of new 
weapons systems, the Navy must decide if the risks of 
unplanned cost growth should deter extending as-yet 
undeveloped technolopy. Prior contractor researen 


primarily by the Rand and General Research 
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wo 


Corporations, has made progress MimeGcri ning and 
Measuring the extent of technological change in complex 
defense systems, primarily aircraft related. aie se 
prier "“stu@ies “have™ concentrated on the generation of 
cost models for specific applications under Department 
of Defense (DOD) contracts, rather than a defined cost 
model for general use by Navy systems commands. For 
exemple, Dr. Ee Dedson developed the High-Energy 
Laser Systems Cost Model (Caliper II) for the Air Force 
Weapons Laboratory at Kirkland Air Force Base, New 
Mexico. [Reimer | However, the Navy has-~ yet to 
coordinate new cost estimating models with the Air 
rorce . 

Cost models which could accurately delineate the 
association between levels of SOA extensions and cost 
overruns would enhance budgeting accuracy. Most budget 
analysts sor multimillion dollar weapons systems 
utilize the Work Breakdown Structure (WBS) authorized 
by the DOD Cost/Schedule Control Systems Criteria. 


[Ref. 3] Effective, comprehensive cost models are 
required for gta eke gl hardware GdeveEropmetr “COS t 
Paemiates, particularly in the constantly changing 
electironlcs environment. The Navy’s negotiating 


capabilities during acquisition for new technological 
~ereenice Will be strengthened with increased expertise 
in SOA extension resource estimation. 

One field rapidly gaining in importance and 
magnitude is software cost estimating, which experts 
like Elmer Branyan from General Electric predict will 
account for Cl onty—-11ve™= = per cent of all embedded 
computer costs by 1990. [Ref. 4] Defense cost models 
must assimilate and utilize the latest techniques 


Lnconmorated — in software models developed by RCA 


(Price-S model) and Hughes’ Aircraft (developed by Dr. 
Jensen). 
Hardware cost estimation for initial developed 


relies on three primary tools: estimation by analogy, 


use of cost estimating relationships (parametric 
costing), and systems engineered analysis (detailed 
bottoms up estimation). These three methods are 


applied on the basis of available data, which will be 
differentiated and described in Chapter Two. 
The following quotation by Larry Smith emphasizes 


the fundamental nature of accurate estimating: 


The process of estimating is the process of making a 


Predilemion Of (lene cas toi redefined events and/or 
occurrences weighted and influence@, by subjectitze 
and objective information. The planning techniques 


that are available rely on the estimates to devel. 
schedules, Tesource Hhistorgrans, Pi eC Se cash flow 
histograms, and performance standards. The resuiee 
and sophisticated methodology are only_as good as 
the input estimates. The time spent developing soga 
estimates and the understanding required to preduce 
good estimates, will reduce the need for  Mmajer 
revisions to the plan and schedule. Good estimates 
will reduce schedule Se aoe and cost overruns 
and set the stage to facilitate the implementationgen 
Project plans: Ref. 5 | 


CC. SOBJEC TIVE 

The objective of this thesis is to describe and 
analyze how defense contractors control the cost of 
states-of-the-art extensions. Litton Applied 
Technology, located in San Jose, California will be the 
Subject of the = case sctudy: This thesis compares 
Litton’s development cost estimate of the AN/ALR-67 
Radar Warning System with their actual development 
costs. Also, Litton’s methodology for quantifying the 
SOA extension to the AN/ALR-67 Radar Warning System is 
analyzed. | 

The following factors integral to Litttentes 
management of the AN/ALR-67 Radar Warning System are 
also discussed: 
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dae Meee omesn cere CSermation process for each of the 
following acquisition phases: concept exploration 
demonstration and validation, full-scale 
devedtlopment, yaad initial production. Pie 
process description will indicate which cost 
become ues Were=utwriZeamby Litton for each 
phase. 

2. Litton’s use of modern parametric models in 
hardware and software cost estimation. 

3. Soundness and range of Litton’s control system in 
Mom toring and reducing cost, schedule, and 
performance variances. 

4. Specific variance analysis by phase, 
Geom@entratimg on the effectiveness of Litton’s 
cost model. 

i Hmpernal factors impacting Litton’s current 
initial production, such as the current GAO 
investigation of commonality problems between 
Navy and Air Force radar warning receivers. 

Oe "OmeaniZA7ational design and interaction of the 
primary departments involved in the SOA 
erauen sion . 

A Key secondary objective, as discussed in Chapter 
Iwo, is to clearly delineate the history of previous 
research of states-of-the-art extension questions. The 
fundamental theory and key elements of early SOA 
measurement equations from 1965 to 1985 are outlined. 
Examples of SOA model applications wOomaireciar t Curbine 
engines, ground combat surveillance radars, computers, 


and laser weapons systems is presented. 


eter oe ARCH QUESTION 
The primary research question examined was: how 
Ccireettmewy did wbittom estimate the development and 


eel 


production costs of the state-ol-the-art ye wee te mem 


the AN/ALR-67 Radar Warning System? Subsidiary 
questions will deal with Litton’s budgetary models for 
predicting and controlling costs. Their methods von: 


cost control are alsomexploredr 
Three subsidiary questions were: 

1. What previous research has been conducredson 
weapon system’s state-of-the-art extensions? 

2. Which cost estimating relationships were 
utilized in Litton’s coSt estimation or sene 
development program? 

3. Was there an association between levels of SOA 


extension and cost overruns? 


E. SCOPE AND LIMITATIONS 

This thesis concentrates on two specific areas. 
First, Litton’s cost estimation process in measure 
the state-of-the-art advance from the AN/ALR-45 to the 
AN/ALR-67 Radar Warning System is thoroughly discussed. 
De COma Uline SOA measurement theory, control and 


planning techniques for defense contractors involved in 


research and development programs is analyzed. For 
example, tie general nature of the S-curve 
described. The S-curve represents how technology 


initially advances’ slowly, then rapidly gains momentum 
and eventually slows down when it nears’7 the natural 
limits of the technology. [Ref. 6] Contractors must 
be able to accurately locate théir exact posit rons tea 
the CurErve- to determine applicability of SOA 
extensions: 

In summary, the case study describes, then  acssecce. 
Litton’s ability to accurately ep redt et eee ome 
development and production costs for the AN/ALR-67 


Radar Warning DyYovem. Peripheral information is 
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PuOVwrGeaGeas review factors in the organization and 
budget process which contribute to the firm’s success. 

The principal limitation of this case study is that 
tineane is no guarantee that observations are 
meanstemeable to another environment, setting, project, 
or firm. For example, the unique mission of fighter 
aircraft radar warning systems may negate the transfer 
ability of finds to commercial aircraft manufacturing 
finms. Also, one sample may not Bem ly 
representative of shipbuilding projects, the primary 
Poews Of LiItton’s Other defense divisions. 

Litton Applied Technology placed restrictions on 
availability of the exact algorithms utilized in their 
cost-estimation model. Specific cost data which might 
reveal overhead rates or management reserve factors to 
their competitors was withheld. Also, Etec Os L 
estimator primarily responsible for initial development 
costs during the 1975-79 time frame was not available 
Poe ater wrews. tne rationale behind the initial cost 
estimates was not clearly documented, SO many 
observations of the SOA extension will be based on 
secondary sources. Additionally, due LEO) IE) dics 
im@classified mature of this thesis, many current 
mission and performance parameters of the AN/ALR-67 


Radar Warning System are omitted. 


F. LITERATURE REVIEW AND METHODOLOGY 

Extensive research has been conducted on methods of 
cost estimation for weapons systems. The majority of 
the literature is based on parametric models, both for 
hardware and software cost estimation. Significantly 
less literature, however, is available on measurement 
of state-of-the-art advances. Knowledge about the 


critical process of high technology cost estimation 


di 


will enable amore credible analysis of Litton’s cost 
estimation procedures for the AN/ALR-67 Radar Warning 
System. This case study can then eC€xplore thew tcnia® 
which theoretical research from the past two decades is 


practically “uvabizedr if at all, by ome of our 
country’s leading defense electronics manuiacimre 
For SOA extensions background literawere- tive 
following key authors and their specialties Mare 
biboh Powe: 
1. A.A. Alexander: estimation of advanced technology 


in turbine enoines? 

2. E.N. Dodson: quantitative measurement of state- 
of-the-art through use of planar and ellipsoid 
SUrhaces. 

oe Richard Foster: analysis of the S-curve, or 
GompertZ curve, scharacterist ice 

4. Results from the 1983 Technological Forecasting 
Conierence which focused yone ni searea- 

The methodology for this thesis primarily was a 
case study of Litton Applied Technology, Inc.) invwsan 


Jose, California. Specifically, personal interviews 
were conducted over a period of two months with the top 
managers involved in the AN/ALR-67 development. Among 


the key company personnel interviewed were Eugene E. 
Deimling, Director of Business Development, and Donald 
R. Bowden, Director of Cost Estimating and Analysis. 
Personal interviews were also conducted with Dr. Edward 


Dodson, Director, Economic Resources and Planning Group 


of General Research Corporation, Santa Barbara, 
California, and Stan Swales, Cost Estimator for GTE 
Government Systems, Inc., MountainwVitew, Califtommia, 


Searches through the Naval Postgraduate Semool 
Library and the Defensem Lé@@istves "s)s vee Woeristiz. 
Exehange were also Veonductede The primary journals 
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ime sthaioa ted were NG ae alee tl Management", IEEE 
Transactions on Engineering Management", and 
Teecmlotropical Forecasting and Social Change". Also, 
Siremt articles ordered from the librarian of the 
Space Systems Cost Analysts Group and the International 
society of Parametric Analysts, Mr. Clyde Perry, were 
rev rewed . 

Finally, titwmeleuOne ss ericing and Cost Estimation 
Manual was examined, with particular emphasis’ on 


procedures for state-of-the-art developments. 


G. ORGANIZATION OF STUDY 
The remaining chapters of this thesis are organized 
as follows: 

ix, Chapter Two: Literature review of previous 
studies of state-of-the-extensions measurement 
theory is presented. Also, comparisons between 
various hardware and software cost estimating 
models are made. Mechanisms espoused in current 
literature for controlling high technology 
development projects is briefly outlined. 

a. Ciapeecr Three: A historical narrative of two 
cost estimation models by Dr. E.N. Dodson is 
presented. Examples will depict cost elements 
which have factored into them SOA technological 
factors. 

Oo. S8@mapter Four: The actual cost data from the 
AN/ALR-67 is presented, along with a description 
Oper tton’s pricing and cost estimation systen. 
Interviews with key individuals are addressed. 

4. Chapter Five: The analysis of variances found in 
tic cost data, and external events which have 
impacted Litton’s development of the AN/ALR-67 
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are discussed. Comparisons from the case study 


to relevant theory are made. 


Chapter Six: Conclusions and recommendations 
developed in this case study are concisely 
stated. Areas if Olle future research are 
identi lea. 
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Pi ee ik ARE EV LEW 


ae ENT RODUCTION 


This chapter explores the background literature 
PJ rtcheatO measurement and control of state-of-the- 


art advances. In particular, a chronological outline 
of SOA measurement history from 1969 to 1985 is 
(OURS Sol ererel The chapter is subdivided into the 
following Key areas: 
ies Ostwald’s basic guidelines for measuring 
technological advances and cost estimating 
relationships. 
2. Dodson’s approach to quantitative measurement of 
advances in state-of-the-art in January 1969. 
S Dodson’s studies in resource estimation for 
development programs from October 1969. 
4. Rand Corporation’s study of measurement of 


technological change in aircraft turbine engines 
(June 1972). 

5. Hovanssian’s description of key parameters 
integral to research and development of large- 
scale electronic systems (August 1975). 


6. Dodson’s development of cost equations to measure 
technological change in high-technology systems 
(May 1977). 

a. Gordon and Munson’s proposed convention for 
measuring the state-of-the-art of products 
(1981). 

8. Alexander and Mitchell’s measurement of 


technological change of heterogeneous products 
(1985). | 


if 


9. Martino’s measurement of  technollor, asa 
tradeoff surfaces (1985). 

10. Dodson’s measurement of SOA and technological 
advance (1985). 

11. Cooley, Hehmeyer,and Sweeney’s model of research 
and development resource allocations (1986). 

12. McDonough’s identification of effective 
characteristics of management control systems of 
new product development projects (1984). 

13. Smith’s summarization of best techniques for 
estimating time, cost, and resources in new 
developments (1985). 

14. Evaluation of the key hardware and software cost 
estimation models used in advanced technology 
development and production contracts. 


B. OSTWALD’S BASIC TOOLS FOR COST ESTIMATING 

Ostwald’s "Cost Estimating for Engineering and 
Management", published in 1974, provides the essential 
basis for detailed, practical cost estimates. [Ref. 7] 
Stan Swales, cost estimator at GTE, recommended 
Ostwald’s book as a prerequisite to analyses of current 
cost estimating techniques. Ostwald details the first 
unified treatment Sof themeyphilosophnyeaecovccpts, soma 
practices of the cost estimating fiveva- The 
fundamentals of three ideas pertinent to later studies 
of SOA advance are introduced: cost estimating 
relationships (CER’s), cost indices, and technological 
forecasting. 

Ve Cost Estimating Relationships 

Ostwald describes CER’s as functional models 

that mathematically describe the costs of components as 
functions of one or more independent variables. CER’s 


are used to estimate physical quantities such as number 
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Of Tra@ars or personnel. CER’s also can express rates 
of activity for support personnel as a function of the 
Himeser Ol —SGQirect radar Operators, for life cycle cost 
Studies. In aerospace electronic industries, CER’s can 
be used to estimate turbojet engine development cost as 
a function of maximum thrust and production quantity. 


mr ee mwuUmdamentLar elements must be present in CER’s: 


a. Logical relationship of the variable to cost, 

b. Statistical significance of the variables’ 
CoMrEri DUtLTOnN:, and | 

& Independence of variables to the cost 
explanation. 


ae Technological Forecasting 
Ostwald defines technological forecasting as 


“Keotca | analyses that leads to quantitative 
conclusions about future engineering qualities and 
properties." [Ref. 8] He recommends evaluation of 
technology trends to find the critical independent 
variables on which the dependent variables rely. 
Correct analysis of the S curve is an essential 
ingredient. The S curve is a feature of nature that 


diagrams how electronic components are subject to flat 
and slow growth in the early years, rapid middle-life 
development, followed by stability and eventually 
decline as new products’ evolve. General technical 
history books provide back-up documentation. 
om Cost Indices 

Ostwald also describes how cost indices 
enable estimators to forecast the cost of new designs 
based on similar items in the past, without going 
through detailed "bottoms-up" costing. The cost index 
is a dimensionless number for a given year, indicating 


the cost of that year relative to a base year. The 


i 


shows the conversion from costs to equivalent costs in 
the present. 


C. DODSON’S APPROACH TO QUANTITATIVE MEASUREMENT OF 

ADVANCES IN STATE-OF-THE-ART (JANUARY 1969) 

In 1969, Dr. E.N. Dodson defined state-of-the-art 
in concept as the highest degree of technical 
accomplishment that could be achieved at any point in 
time. [Ref. 9] This was later revised to represent the 
state of best implemented technology. This early study 
focused on parameters which comprehensively influenced 
achievable engineering designs. 

Dr. Dodson stated that the following constraints 
must be met before SOA advances could be quantitatively 
measured: 

1. A geometrical surface with continuous derivatives 
that interpolates between implemented design 
characteristics is used to approximate the true 
state-of-the-art. 

2. The surface should be either convex or concave in 
all dimensions, permitting plane surfaces. 

3. Each pair of design characteristics must be 
negatively correlated to allow tradeoffs on 
single state-of-the-art surfaces. 

Considering these constraints, Dr. Dodson asserted 
that a convex hypersurface reasonably represented SOA 
surfaces. Actual data points are fitted to convex 
surfaces in ellipsoid geometric forms via mathematical 
relationships. The chosen physical or performance 
characteristics must be among those derived early in 
the concept exploration acquisition stage. These 
characteristics should be influenced by engineering 
development decisions’ to have relevance to SOA 


determination. Also, Dire Dodson stipulated that 
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characteristics should be specified so that increasing 


values corresponded with increasing mechnical 
aQTffricul tye 

Dr. Dodson worked with subsystems of solid 
propellant missiles, maintaining the Department of 
Defense Work Breakdown Structure. Three parameters of 
propulsion subsystems which net his mandatory 


prerequisites were: delivered specific impulse, 
propellant mass fraction, and length-to-diameter ratio. 
These parameters were considered the primary, but not 
all ime lWsive, factors which represented tee 
technological advance. These variables influenced 
areas of SOA relevance such as’ variations in chamber 
pressure and burning rate. Dr. Dodson then assigned 
values to these parameters based on the Chemical 
Propulsion Information Agency Rocket Motor Manual. His 
next action was to classify these motors’ in various 
time periods according to year of development 
completion. 

Dr. Dodson’s purpose in quantifying measurements of 
advances in the SOA was to test the hypothesis that SOA 


extensions influenced the costs Peg rem for 
development programs. Through geometric methods, he 
based SOA measurement advances upon how data points fit 
above existing SOA surfaces. The data points 
represented the chosen parameters, classified by year 
of dewe bopment. The measured SOA advance becomes a 


Pilea the proportionate increase in the radial 
draway from the origin through the surface to the new 
data point. The SOA advance is indicated by the 
squared proportional increase. 

Dr. Dodson summed up his 1969 study by stating that 
the measured SOA advance is comprised of these two 


factors: 


Zo 


1. Attributes of design efforts directly Vetarea. ce 
the particular system development program, and 

2. Contributions from general research ang 
development factors, which can be represented by 
dummy variables identified for specified major 
technological advances. 


D. DODSON’S STUDY OF RESOURCE ESTIMATION IN 

DEVELOPMENT PROJECTS 

Dr. Dodson, ina study for the Assistant Secretary 
of Defense, elaborated his earlier study on SOA 
advance. He described the SOA for a particular point 
in time, with n representing the number of SOA désten 
characteristics. These characteristics could trade off 
upon one another. Some examples are weight, speed, and 
size for fighter aircraft’s SOA. 

Dr. Dodson indicated that Work Breakdown Structures 
should be used to divide whole weapons systems, such as 
missiles, into subsystems. SOA specification for an 
aggregate system would not be reliable since subsystem 
technologies are distinct and will advance 
independently of one another. For example, missile 
propulsion and guidance subsystems evolved separately, 
with different primary parameters. He recognized that 
information about performance, development costs, and 
development time of subsystem is severely limited, due 
to contractor reluctance to divulge this information. 


In this reéeporme, Dre Dodson concentrated on 
specifying SOA measurement equations for inertial 
guidance systems for missiles. Accuracy, weight, and 
reliability were selectCdmassagime prudmeipal parameter. 
affecting SOA. These three factors could be 
quantified. They explained) based om engineering 


Cie 


judgment, most of the resources spent on advancing the 
SOA. 

The measure of SOA for accuracy was maximum range 
divided by Circular Error Probable {CEP} for inertial 
guidance systems. CEP represented the acceptable 
frscomree a Missile must steer to its target. This 
distance could be fractions of miles for a nuclear- 
armed ballistic missile. Since state-of-the-art 
advance always increases, the above equation indicated 
that, as range increased, given accuracy became more 
Giftficult to achieve. 

The measure of SOA for weight was the number one 
Givided by the guidance system weight. Dr. Dodson’s 
study found light guidance systems outperformed heavy 
ones since they permitted greater range and payload. 

The SOA parameter for reliability was mean-time- 
between-failure (MTBF). For inertial guidance systems, 
refrabvility was critical due to the continuous state of 
twenty-four hours a day alert status. 

Dr. Dodson indicated several reasons why production 
cost was not a good choice for an SOA parameter: 

alles: Costs prior to development would be derived from 
cost estimating relationships, which would not be 
teestorical yy reliable. 

2. Learning curves could not be established before 
actual engineering efforts had been expanded to 
reguce costs. 

Pinallyy by geometric measures of the data points 
for accuracy, weight, and reliability on an ellipsoid 


surface, SOA advance indices were developed. Some SOA 
index numbers were 2.945 for the Pershing missile, and 
mo coU0stor . the "Polaris missile. Research using the 


SOA parameters demonstrated Pershing had a relatively 


low SOA advance for two reasons: 
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1. Relatively low accuracy at maximum range, and 
a Low reltabiaevey. 

Fingal] y aebir. Dodson devised a logarithmic approach 
to measure the cost of an SOA advance, in relation to 
time. His basic hypothesis stated that time and cost 
are interrelatved Tesource-.. He developed an equation 
treating the cumulative cost to complete development 
(C) as a dependent variable, with the measured advance 
in SOA (S) as an independent variable. The elapsed 
time to development completion (T) would be stipulated 
by the planner. The equation took the form of: 

log C = a+b log S&S + Galocem.: 

where a,b, and c were designated parameters, based 
on the system. 

The next section outlines how Rand Corporation 


expanded on Dr. Dodson’s approach. 


E. MEASURING TECHNOLOGICAL CHANGE IN AIRCRAFT TURBINE 


ENGINES 

Alexander and Nelson from Rand Corporation 
developed techniques in 1972 to measure technological 
advance in weapons acquisition. [Ref. 10] Their 


initial studies focused on aircraft turbine engines. 
Their goal was to capture mainstream trends and improve 
estimates of costs and schedules during the system 
acquisition cycle. 

Alexander and Nelson used multiple regression 
analysis to develop an equation containing the primary 
parameters important in turbine engine technological 
change. Like Dodson previously, they estimated 
multidimensional tradeoff surfaces of the key turbine 
engine parame ters. Using regression analysis, they 
traced out the movement of the tradeoff surface over 
time. 
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The equation they developed to calculate indices 
for turbine engine development was: 

Tech = -1187.5 + 156 Ln Temp + 18.8 Ln Thrust- 

Poe tn Weleht — 20.6 Ln SFC + 11.7 Ln @ + 13.0 

Prop. 

The dependent variable represented the technology 
index, Which was measured in quarters of a year 
beginning in January 1943. The independent variables 
meoprmesen Led™were: 

i. Temp = turbime inlet temperature in degrees 
Rankine. 
ew Thrust 


pounds. 


miei tary sea level static thrust in 


Weight = engine weight in pounds. 


a. oFC = specific fuel consumption at military sea 
level staticthrust (lb/hr/lb). 

om Q = maximum dynamic pressure in pounds per square 
foot. 

6. Prop = dummy variable, equal to one if the engine 


was a turboshaft or turboprop, zero or otherwise. 

Alexander and Nelson found that the major turbine 
creme Manufacturers had similar tradeoff surface 
Shapes. However, the two major manufacturers, General 
Electric and Pratt and Whitney, were approximately two 
years ahead of Liege ile competitors in level of 
technology. 

Alexander and Nelson specifically addressed the 
question: "Can a technique be developed for objectively 
quantifying the technological state of the art of a 
particular type of system?" [Ref. 11] Their analysis 
utilized the following two assumptions: 

1. Limited numbers of parameters adequately 


characterized the system under study. 
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2. Historical continuity prevailed such that the 
selected parameters characterized the system even 
during different time periodse 

Alexander and Nelson also recommended the use of 
performance parameters ra titer than teenniecal 
parameters, to give greater emphasis to outputs than 
nM e Ss . However eeu y acknowledged their final 
equation contained both technical and performance 
parameters. Their data used in their study came from 
engine manufacturers, and standard sources’ such as 
"Jane’s All the World’s Atreranre. Turbine engines 
were well-suited for analysis due to their strong 
technological trends, such as the increacewrmmesairecrar & 
speeds and the progression of engine types. 

Alexander and Nelson subjected the data to 
statistical tests with different subdivisions to find 
the equation with the best "fit" over the tradeoff 
surface. However, they did not develop ' specific 
equations for obtaining the cost of development of the 
technological advance. They hypothesized 
quantification of the technological setting were 
necessary prerequisites, before the cost question could 
be answered. 

In the next section, Hovassian’s Key Parameters 
necessary for development of electronic systems are 
examined. 


F. HOVANSSIAN’S DESCRIPTION OF KEY PARAMETERS 
Hovanssian described the research and development 
phase of a hypothetical large-scale electronic systen, 
based on his study of applicable programs. [Ref. 12] 
He stated that the initial design phase should always 
include a state-of-the-art analysis of the key system 


design parameters. This analysis should include a risk 
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evalWwation™ of pemwentlal prop lem areas Such as 
technology, cost, or time. He stated that the more 
current performance measures are advanced, the greater 
the risk imwvolved. 

Hovanssian did not develop any models to estimate 
the SOA advance. However, many of his cost estimating 
techniques matched those of Litton Applied Technology. 
Pretem s cos® model is described in Chapter Four. 

Hovanssian stated that the cost-estimating 
relationship technique should only be used during early 
stages of development. He recommended that the cost- 
estimating relationships be based on statistical 
analysis of similar equipment. If statistical data on 
similar equipment was not available, thew he 
recommended the utilization of "bottoms-up" system- 
level cost estimating. In systems-level estimating, 
costs are derived from consideration of manpower, basic 
units, components, parts, or other relevant factors. 

Hovanssian also described "design-to-cost" 
modeling. In this Lechmiigue , various design 
configurations are tested against specified performance 
parameters. The successful design configurations are 
traded off to determine which yields the lowest life- 
cycle cost. Life-cycle costs included both the total 
procurement cost of the system and its future operating 
costs. 

kinallg , Hovanssian stated that most electronic 
systems must include customer-acceptance parameters at 
the development stage, or face possible cost overruns 
Paver in the life-cycle. Among the parameters listed 
were: 

Nis Lite eyele cost. 
2. Maintenance skill required to repair the system. 


ZAM 


83 Number of maintenance employees required to keep 


the system in working order. 


4. Percentage of time the system is available for 
use. 

5. Reliability (measure elapsed time between two 
consecutive failures). 

6. Amount of maintenance required per operating 
hour. 


7. Quantity and type of spare parts required. 
Operator approval of new system. 
Degree of automation. 

10. Improvement over previous systems. 

In summary, Hovanssian’s paper pointed out many 
parameters and fundamental concepts which entered into 
the design and development process for large electronic 
systems. In the next section, Dr. Dodson’s updated 
1977 study on quantification of SOA advance is 
outlined. 


G. DODSON’S DEVELOPMENT OF COST EQUATIONS TO MEASURE 

TECHNOLOGICAL CHANGE 

In May 1977, Dr. Dodson outlined procedures to 
account for technological change via research and 
development cost equations. He indicated research and 
development costs were directly influenced by the 
degree of extension to the SOA by a given program. 
[Ref nace His measurement approach to the SOA advance 
evolved from a combination of his earlier study 
(discussed in section C) and Alexander and Nelson’s 
paper (discussed in section D). Dr. Dodson felt this 
new approach would be easier to implement. 

This approach featured four key elements: 


is Independent variables for a multiple regression 
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equation would consist of a limited number of 
key technical parameters. 

2. The dependent variable would be a consistently 
defined calendar milestone. For example, a 
completion of qualification test would 
represent this milestone for applicable 
subsystems. 

3. An expected date (Ye) for achievement, for the 
parameters designated as independent variables, 
would be the result of a multiple regression 
excrewsce. 

4. The residuals of the difference between the 
expected end the actual date of achievement 
(Y estimated - Y actual), would be the measure of 
the relative technological advance. 

Dr. Dodson illustrated his theory through his 
previous research on avionics computers. The 


computers’ key parameters were selected from technical 


data as: 
1. The speed of the computer in operations per 
second. 
2. The density of the central processing unit in 


pounds per cubic foot. 
3. The number of distinct instruction types in the 
computer. 
Dr. Dodson also documented the actual date of 
development completion for each computer as Y actual. 


The difference between the expected and actual dates of 


completion represented the change in SOA, in units of 
years. 

Ome the residual was caleulaced, it was 
incorporated into a cost estimating relationship. ae 


dependent variable would represent the development cost 
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associated with the specific SOA advance. Tite 
independent variables were: 

1. X1, which represented the residual difference 
between the expected and actual dates of 
development completion. 

2. X2, which equalled one for microprogrammable 
computers, or zero for synchronous computers. 

3. X3, which equalled one for space computers, or 
zero for airborne computers. 

The resulting equation developed by Dr. Dodson to 
calculate the monetary amount correlated with SOA 
advance was: 

Y = 6.11 + 2.7 X1 - 4.57 X2 + 14.8 X3 

Drs Dodson recognized -‘:several limitations to his 
SOA measurement approach. The equation might simply 
reflect a change of mission requirements, rather than 
indicate technological advance. To remedy this 
potential problem, he recommended that variables should 
be stated via efficiency measures like thrust per 
pound, rather than absolute-scale measures like thrust. 

Dr. Dodson also recommended in this paper that 
electronics development firms should develop an output 
Linde ss to account for year-to-year changes’ in 
manufacturing productivity. He felt that failure to 
consider productivity changes contributed to errors in 
estimating electronics procurement costs. Two reasons 
for productivity changes were: 

1. Changes in design and manufacturing technology 
which reduce the cost per unit of output. 

2. Operational requirements for performance 
increases. 

Using costs as the dependent variable, cost 
estimating relationships were derived by Dr. Dodson for 
the technology output yindice.- [in Stibismeace se ENe vear 
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of development completion became an independent 
variable. The regression coefficient represented the 
annual change in costs due to shifts in the 
relationship between productivity and cost. 

In summary, Dr. Dodson developed a method _ to 
measure the financial impact of SOA advance. In the 
next section, Gordon and Munson proposed an alternate 
method for measuring the SOA advance. 


H. GORDON AND MUNSON’S PROPOSED CONVENTION FOR 

MEASURING THE SOA OF PRODUCTS 

In 1981, Gordon and Munson developed for’ the 
National Science Foundation a general equation to 
measure SOA advance. [Ref. 14] The equation derived 
was: 

sem — i (P1/P2’) +*2~eP2y/Pp2’) +... Kn (Pn/Pn’ ). 
In the above equation Kn represented the relative 
weight associated with each parameter describing the 
technology. Pn indicated the value of the parameter 
chosen to describe the SOA, while Pn’ represented the 
maximum reference value of the parameter. 

Gordon and Munson developed their SOA equation to 
satisfy the following criteria: 

1. The same estimate of SOA should be produced by 
any analyst studying the same technology. 

2s The SOA value should be in an index format, based 
@naw®m®eferenece vwadkue. 

3. The equation should satisfy any level of 
technological aggregation. 

Gordon and Munson discussed two techniques critical 
to the choice of parameters and their relative weights: 
Judgmental and statistical techniques. They 
recommended Delphi-related procedures, the utilization 


of experts to choose the proper parameters for the SOA 


on 


equation. Delphi» procedures souvent expert Judgement via 
anonymity and controlled feedback, to obtain the most 
objective inputs. For example, electronic voting 
devices were utilized to allow expert practitioners 
freedom to provide anonymous answers to questions posed 
by group moderators. 

Gordon and Munson also listed several statistical 
methods available to estimate weights and parameters of 
the SOA equation. The techniques included: multiple 
regression, stepwise regression, discriminate analysis, 
cluster analysis, multidimensional scaling, and factor 
analysis. ifn factor analysis, for example, a 
correlation matrix was utilized to group variables into 
classes of highly correlated components. These classes 
were taken as parameters through construction of an 
index from individual members of the class. 

Gordon and Munson applied their theory against 
antibiotics and computers. They formulated a 
performance index for computers based on the judgments 
of an expert panel using the Delphi approach. The 


following three parameters were chosen to characterize 


computers: 
1. Computer speed by operations per second. 
2. Cost of computation by operations per dollar. 


3. Maximum memory size by kilobytes. 

Gordon and Munson used computerized statistical 
techniques, based on their assumption that the state- 
of-the-art function is an S-shaped curve. Using their 
results, they developed an SOA index for computers, 
ranking the IBM 3033 computer as the SOA leader. 

Gordon and Munson felt their process of calculating 
sOA measumess scowld be applicable to amy a re ld , 


including integprateu™ circu hye heVtewewceecrron , 
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Alexander and Mitchell’s study of technological change 
of heterogeneous products is outlined. 


I. ALEXANDER AND MITCHELL’S MEASUREMENT OF 

TECHNOLOGICAL CHANGE 

In 9S 5, Alexander and Mitchell developed 
procedures to measure technological change of 
heterogeneous products. [Ref. 15] They designed a 
framework in which to place empirical measures of 
technological change, hedonic price indices, and cost 
estimating relationships. In this study they estimated 
technological change equations for milling machines, 
turbine-powered airliners, and turbine engines. Like 
Peason Carlier, they realized product characteristics 
alone could not define technological change. 

Alexander and Mitchell derived a tradeoff function, 
based on a tradeoff surface of cost, performance, 
technical characteristics and time. The entire 


tradeoff surface represented the state-of-the-art of a 


given period. The tradeoff function was written as: 
mOMORH. oc PS, Ghee wn GQ). tt). 
The variables in this function were: 
a "C" = average cost of the product. 
Os pl performance characteristics or user 
outputs. 
"q" = factor prices (assumed to be fixed). 
4. =” = time. 
Their equation meant technological change in 


products arose from three factors: 


die Productivity or performance improvements. 
Bk, Improved factor inputs. 
3. Production process improvements. 


Alexander and Mitchell noted that their equation 
took the form of the typical cost estimating 
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relationship, with cost as the dependent variable. The 
dimensionality of the state-of-the-art surface was 
limited by the difference between the total number of 
variables and the number of constraints. Mred Pr om@s tk: 
important reason for including costs in the measure of 
technological change was their belief that higher 
levels of performance were attained through the 
expenditure of more resources. 

Alexander and Mitchell studied the technological 
histories of milling machines, airframes, and aircraft 
turbine engines, summariZinoewrnelr  speedticepeca tom 
this paper. They concluded that measures based on 
product characteristics must be evaluated carefully, 
since selection of a few core characteristics often 
neglects other attributes whose relative importance my 
have changed over time.) rey telr prodadmcet i view, 
measures based on user outputs, such as cost per mile 
for aircraft, were more likely to capture the totality 
of technological change. 

In the next section, Martino’s studies, which also 
concentrated on measurement of SOA surfaces, are 
highlighted. 


J. MARTINO’S MEASUREMENT OF TECHNOLOGY USING TRADEOFF 

SURFACES 

In 1985, Joseph Martino followed up Alexander’s 
theoretical derivation of the designer’s tradeoff 
surface with an empirical approach. He concentrated 
only on those technical parameters which were relevant 
to measuring the SOA. He viewed the SOA as a "surface 
in some multidimensional parameter space." [Ref. 16] 

Martino stated that an SOA advance was a 
prerequisite before designers could move to higher 


surfaces. He reasoned that designers were constrained 
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by technology and economics to stay on the SOA surface. 
AS a result, improving the value of one technical 
parameter, such as speed, meant sacrifice of another 
parameter, such as weight. He examined several 
technologies to see if the data values on a surface had 
a discernible or random pattern. His goal was to 
determine the actual variables involved in design 
tradeoffs. 

Martino’s theories drew heavily from Dodson’s work 
except for two statistical differences: 

1. Martino extended his method to allow ellipsoid 
surfaces of any order, not just level two. 

2. Martino’s surface fitting procedure was changed 
to minimize the Mean Absolute Deviation rather 
than Mean Square Deviation. 

He felt Dodson’s fitting method was prejudiced by 
Cmrreme date values. 

Martino applied his surface-fitting procedures to 
clipper sailing ships, jet engines, propeller-driven 


aererarc, and power transistors. In selection of 
variables, he avoided the problem of scale effects by 
using nondimensional variables. For example, he 
divided a size variable by a "characteristic length" to 
Cancel out scale effects. He also utilized "data 
triplets", where two positively correlated variables 


negatively correlated with a third variable. 
Marcino’s primary results were that® 
1. Deviations from the SOA surface could be 
explained quantitatively by known 
enaracteristics. 
2. Different configurations representing the same 
SOA belonged on the same surface. 


3. An engineering analysis was necessary in order 
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to seleCte@ihe proper varrapkes: Designers were 
asked what parameters they worked on to improve. 
4. The tradeoff surface technique worked for all 
levels of the Work Breakdown Structure. 
In the next section, Dr. Dodson’s 1985 update of 
his SOA measurement theory is discussed. 


K. DODSON’S MEASUREMENT OF .SOA AND TECHNOLOGICAL 

ADVANCE 

Dr. Dodson’s paper for the Workshop on Technology 
Measurement, Dayton, Ohio (12-14 October 1983), was 
published in 1985. [Ref. 17] This paper summarized his 
earlier works from 1969 to 1977. Also, he included two 
other approaches used to measure SOA advance. 

Dodson described the factor analysis approach, a 
statistical technique for analyzing variance. He felt 
that factor analysis would help in identifying the 
underlying relationships of the many physical and 
performance characteristics of components. For 
example, he used factor analysis to rank sixty rocket 
motors by "Technological Distance Scores". He gathered 
data from the Chemical Propulsion [Information Agency 
Rocket Manual for the following nine variables: 

1. Delivered specific impulse. 
Mass ratio (propellant weight/ motor weight). 
Length-to-diameter ratio. 
Reciprocal of burn time. 
Motor weight. 
Average thrust/ burn time squared. 
Average thrust. 


Average chamber pressure. 
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Date of development completion. 
The factor analysis for the nine variables was able 


to account for eighty-three percent of the total 
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variance of these variables. From the computed 
Technological Distance Scores, Dodson demonstrated 
graphically in this paper the effects of changes in 


mission requirements and technological capabilities. 


For example, the mission objectives for the Sprint 
missile required fast burning motors. Overall, this 
missive represented a significant technological 
advance. 


His second new approach to measurement of SOA 
focused on the time available to develop levels of 
technology. His results revealed that for a given 
number of years, the higher the desired technology, the 
@reater the risk. Conversely,the more time available, 
the lower the risk. However, Dodson’s research has not 
extended this variant of the time factor into specific 
examples of SOA measurement. His paper indicates that 
application of this feature to multiparameter SOA 
surface equations must be a subject of future study. 


L. MODELING RESEARCH AND DEVELOPMENT RESOURCE 
ALLOCATION 
Cooley, Hehmeyer, and Sweeney developed the 
Technology Resource Allocation Model (TRAM) to analyze 
the impact of project selection, funding, scheduling, 


technical risk, and staffing upon an organization’s 
research goals. [Ref. 18] Cooley’s model was designed 
to answer what-if questions such as : what performance 
degradation could be expected if a five year 


development period was shortened to three years? The 
important design considerations in this model were: 
realism, model flexibility, ease of use, and output 
format. TRAM produced both tables and graphs which 
plotted the expected progress of the research effort as 


a function of time. 
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TRAM was SUrUC LUIvedaseo allow variability by 

analysts in the srelWow ino rac vOnRs. 

1. Required funding by work unit. 
Actual funding by work unit. 
Probability of success by work unit. 
Engineer manager work load by engineer manager. 
Contribution factor for each work unit. 
schedule extension by work Unit. 
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Performance objectives by work unit. 
Cooley’s model included consideration of the S 
curve effects. S curves normally depicted 
transpiration of relatively long lead times before 
significant results were achieved. 

TRAM enabled the researcher to quickly analyze the 
effects of budget reductions on state-of-the-art 
development projects. Different measures of 
effectiveness were generated by variances of cost, 
schedule, and performance criteria. The sensitivity 
analysis for TRAM was accomplished through use of the 
DYNAMO Compiler. 


M. MANAGEMENT CONTROL OF NEW PRODUCT DEVELOPMENT 
PROJECTS 
McDonough felt management control systems were an 
essential ingredient in the success of state-of-the-art 
development projects. [Ref. 19] His research found 
that management control directly impacted SOA projects 
in the following ways: 
Accuracy of cost and duration estimates. 
Rate of progress of the SOA project. 
Size of the development budget. 
Quality of the output. 
Competitive ability of the organization. 
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McDonough’s paper identified the characteristics of 
euepective management control Systems. He also 
identified deficiencies common to most research and 
development organizations. 

McDonough surveyed twelve large new product 
development projects. He found four elements common to 
all SOA developments. The first element was setting 
goals for new product projects. He found the primary 
goals were cost budgets and schedules. McDonough’s 
results revealed that pressure from the Marketing 
Department forced the majority of project leaders to 
submit unrealistic low cost and duration estimates. He 
also found product specifications were of more concern 
to top management than budget or schedule overages. 


McDonough’s second element was the monitoring of 


project progress. Companies surveyed used _ three 
devices to monitor projects: written reports, formal 
meetings, and informal meetings. He Tound written 


reports and formal meetings had drawbacks in timeliness 
and detail for highly innovative technology-based 
projects. The personal monitoring of projects by top 
management via informal methods was the only way to 
quickly remedy new problems. 

McDonough’s third critical element was management’s 
response to deviations from schedules and budgets. He 
found most companies reluctant to take action on 
technical issues of SOA projects. Their most common 
solution was to simply assign more engineers, an action 
that results indicated rarely provided desired effects. 

McDonough’s fourth element for management control 
systems was incentive provisions for performance. He 
found management rarely tied individual rewards to the 


attainment of budget and schedule goals. 
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In summary, McDonough stated the "management by 
walking around" was the key to successfully controlling 
new SOA projects. He also recommended bonuses to 


project managers who met budgets and schedules. 


N. SUMMARIZATION OF RULES FOR BETTER ESTIMATES 

Ssmith’s paper provided an overall summary on the 
research and application of time, cost, and resource 
estimating. [Ref. 20 | He developed rules for 
generating better estimates applicable to any project. 
The following list presents smith’s key factors 
necessary for the estimating process: 

i Level of Detail: provide a more detailed 
description via increased levels of the work 
breakdown structure. 

ee Precise knowledge of the task being estimated. 

3. Competency and Knowledge of the process being 
estimated. 

4. Importance of the estimate: estimates must 
appreciate significance. 

5. Common units: all cost estimates should reflect 
the same dimensions. 

6. Uncertainty: estimates should indicate measures 
of the maximum possible error. 

7. Assumptions should be explicitly stated. 

8. Uncontrollable variables should be incorporated. 

Smith felt estimates could be improved by use of 
more than one technique, such as estimating by analogy, 


fara quotes, handbook estimating, parametric 
estimating, or regression analysis. He also stated 
that relevant historical data was often overlooked. He 


stressed the importance of detail in the work breakdown 
structure, so errors in one estimate would not have a 


ereat aggregate Cr eec tk. Finally, eeomllth = stated 
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Management could use certified professional estimators 
for uncertain projects, since they would possess 
increased appreciation for the repercussions of 
estimates on the baseline plan. 


O. EVALUATION OF KEY SOFTWARE COST ESTIMATING MODELS 
In 1981, Thibodeau performed the Air Force’s first 
large scale study of the various software cost 
estimating models used to make estimates of the 
resources to be invested in the software subsystems. 


[Ref. 21] Thibodeau recognized that hardware cost 


estimating was more advanced, possessing more 
identifiable measures of size and performance which had 
been correlated with cost. He found there were no 


remranmre procedures for quantitatively describing the 
effects of non-product factors on cost. 


Thibodeau evaluated and provided descriptions for 


the following models: Aerospace, Boeing, DOD Micro 
Estimating, Doty Associates, Tecolote, Wolverton, PRICE 
S, SLIM, and Farr and Zagorski. He provided one page 


summaries for each model type, including descriptions 
of the estimating procedure, characterization of 
productivity, and outputs. 
Thibodeau’s comparison of the outputs indicated 
1. Supporting materials for the models did not 
precisely state the elements included in their 
estimates. 
2. The models were more adept at satisfying 
information early in the acquisition life cycle. 
3. The models were acquisition phase oriented and 
did not describe activities that crossed 
different phases. 
A. Only PRICE S Kept track of the cost ona 


component basis and accounted for the cost of 
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system integration. However, none of the models 
could provide costs for every level called for in 
the Work Breakdown Structure. 


Thibodeau measured performance based on the 
relative root mean square error. He found: 
1. Recalibration was the primary contributor to 


differences in model estimating performance. 

2. The structure of the model was not significant to 
estimating accuracy. 

3. The development environment significantly 
influenced the performance of the cost estimating 
models. 

4. The use of size as an input had no effect on the 
relative performance of the models. 

o. The average root mean square estimating error was 
between fifteen to Ghirityepercen 


In his final section, Thibodeau provided 
recommendations for future model development and better 
data definition and collection. Finally, Thibodeau 


described the derivation of technology and complexity 
factors for each of the nine surveyed cost estimating 


models. 


P. SUMMARY 

Chapter Two described a sampling of background 
literature relevant to measurement and control of SOA 
advance development projects. Chapter sinree presents 
examples of a few specific hardware cost estimating 
models utilized by General Research Corporation and the 
Air. Fonce. 
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Ill. DEVELOPMENT OF COST ESTIMATING MODELS 


A. INTRODUCTION 


Chapter Three describes’ the development of four 
cost estimating models, including three specific ones 
derived exclusively for weapons’ systems. For each 
model the general methodology is highlighted, followed 
by sample cost derivations for various cost elements. 
The four models examined are: 

1. Dodson’s cost estimating models for ground combat 
Surveillance radars (1968). 

as Yodson’s cost estimating methods for the 
High-Energy Laser Systems Cost Model (1979). 
The Unmanned Spacecraft Cost Model (1981). 

4. The Freiman Analysis of Systems Technique (FAST). 


B. GROUND COMBAT SURVEILLANCE RADARS 

Dr. E.N. Dodson developed cost estimating methods 
for ground combat surveillance (GCS) radars during a 
study for the U.S. Army Electronics Command in 1968. 
The basic methodology is still applicable to larger, 


more current system studies. The goal of Dr. Dodson’s 
model was to provide Army comptrollers with a model to 
evaluate contractor cost estimates. [Ref. 22] 


Dodson’s GCS radar model considered state of the 
Siete limitations. He stated that the major objective in 
the design of GCS radars was to achieve minimum weight 
for a specified performance. He used maximum range as 
the crude measure of radar performance. To prevent 
development and production cost penalties, Dodson’s 
study indicated that radar design engineers imposed 
constraints that radars to be costed should not be 
improved by more than five percent over the original 


SOA curve. 
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To derive the model, Dodson first collected data on 
twenty input variables. He utilized the work breakdown 
structure, disaggregating each life cycle element into 
a set of functional subsystems. Ineweiate © cycle 
elements were designated as engineering development, 
advanced production engineering, and production. The 
work breakdown structure for engineering development, 
for example, was broken down further into hardware 
fabrication and documentation. 

In Dodson’s next step, he derived a cost estimating 
relationship for each category in the work breakdown 
structure. The cost estimating functions were derived 
through the following basic steps: 

1. All Known factors between the variables of 
interest were specified. 

2. Regression equations were developed through a 
sequence of known intermediate relationships. 
The mechanisms by which an item’s physical 
characteristics affected raw material quantities 
and labor hours were investigated. Engineering 
information related the functional variables of 
interest to the physical configuration. 

3. Standard curve fitting techniques determined the 
constants associated with the regression 
equations. 

4. The statistical properties of the resulting 
correlation were measured. 

Dodson claimed the model’s results provided no more 
than a basis for judgment, since previously confirmed 
observations were his only data source. Also, he did 
not have enough data to allow estimation of confidence 
intervals for his cost predictions. 

During i tale model’s development stage, Dodson 
gathered cost data from seven different radars. He 
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used a price index developed at General Research 
Corporation to normalize the dollar price data to 1966 
eee |S. His index assumed an equal breakdown of 
emerneering labor, production labor, metals and 
hardware, and electrical equipment. 

Dodson utilized learning curve assumptions’ to 
normalize cost data with respect ar olilimiieecsaye 
production quantities. For both complete radar sets 
and subsystems he used a slope Vaue OL Ninety —per cent. 

The GCS cost model was composed of the aggregate 
estimating relationships for each element. Dodson 
derived production costs first, using the production 
cost per unit as an input variable to the engineering 
models. No more than two or three input variables were 
required to use any one relationship. 

The model input variable for production included: 

1. Type of design, either pulsed or FMCW. 
Number of radars in the first production lot. 


GO 


Type of antenna material, either aluminum or 
fiber glass. 

Antenna frontal area in square feet. 

Center frequency in megahertz. 

Peak radiated power. 


Range resolution. 


Ona oOo fF 


Type of presentation, such as handset, 
loudspeaker, or plotter board. 
Total radar set weight. 

nor Prime power source, either battery or 

motor-generated. 
hie only inputs required for engineering 

development were cumulative average production cost and 
number of development models fabricated. 
Dodson derived advanced production engineering 


costs from the cumulative average production cost, 
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number of prototype models fabricated, and the number 
of radars to be produced in the first production lot. 
The model was built from actual production cost data 


describing the various subsystems. Subsystems included 
antennas, scanheads, microwaves, transmitters, 
receivers, presentation and control, chassis, 


interconnections, casings, and tripods. 

Dodson derived a regression equation to estimate 
the microwave subsystem of a radar. He defined the 
microwave subsystem as a "collection of switching and 
waveguide components that carries microwave signals 
between the transmitter and antenna, and the antenna 
and receiver." [Ref. 23] The key input variables 
researched were frequency and peak radiated power. His 
final model for microwave cost was derived by adding 
the fixed waveguide cost at each frequency to the 
power-dependent duplexer (switching) cost. The cost 
estimating relationship for pulsed microwaves was found 
through regression analysis to be: 

$1100 + $9.4 P, with "P" equal to kilowatts. 

Dodson collected cost estimates for all production 
subsystems, such as the microwave, to obtain total 
subsystem cost. Next, he added on the final assembly 
and test cost to obtain the cumulative average radar 
set cost. The learning curve factor then adjusted the 
total cumulative average radar set cost. The prime 
power cost was added to the previous sum to obtain the 
cumulative average cost for prime power hardware. 
Finally, the above total was multiplied by one hundred 
five percent to account for the additional cost of 
technical data. The result was the cumulative average 
radar production cost. 

Engineering development costs were calculated by 


multiplying the cumulative average radar production 
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costs by a factor of three to get the cumulative 
average cost for development hardware. He based this 
factor of three on the historical trends of the seven 
systems studied and consultation with industry experts. 
The learning curve factor was applied with the number 
of units to obtain the cumulative total development 
hardware cost. An additional fifteen percent was added 


on to account for the extra cost of documentation. 


Dodson’s' model also provided a hierarchy of 
estimating systems, depending on the data available to 
the analyst. He recommended all information sources be 


exhausted before reliance on estimates based solely on 
the regression equations. 
The next section provides a brief overview of the 


laser system cost model developed by Dodson a decade 
lhe. 


C. HIGH-ENERGY LASER SYSTEMS COST MODEL 

In June 1979, Dr. E.N. Dodson completed a study for 
the U.S. Air Force to develop a comprehensive life- 
cycle cost model for high-energy laser weapons systems. 
[Ref. 24] The study accomplished the following 
objectives: 


Jig Created a cost data base for laser weapons 
systems. 
ins Developed cost estimating relationships for laser 


weapons systems. 
ou Integrated the cost estimating relationships into 
a life-cycle cost model. 
4. Used the cost model to project costs for a number 
of weapon system concepts. 
Laser weapons systems were required _ to detect 2 
target and establish its position. This knowledge 
enabled the high-energy laser beams to be properly 
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Caitec ne ar Other system requirements were target 
identification, threat assessment, firing doctrine, 
kill assessment, plus miscellaneous support equipment. 
Dodson relied on the parametric method to develop 
cost-estimating relationships for the laser systems. 
Since laser systems had never been produced previously, 
he used two different approaches’ to gather the 


necessary data. First, he used analogous equipment of 
similar physical complexity for analysis. For example, 
liquid-propellant rocket motors were considered similar 
to a laser device’s mechanical elements. Also, Dodson 
gathered cost estimates for laser subsystems by 
currently active contractors in the field. 

Additional modeling considerations included an 
assessment of mission performance. Performance was 
considered a system parameter measured at the 


aggregated level of equipment detail. He developed 
performance-cost relationships for particular types of 
designs at the aggregate level. Dodson’s model was 
capable of updating the performance-cost relationships 
with changes from new technological developments. 
Examples of performance criteria for lasers were radar 
detection range, laser power, mission kill probability, 
and energy density on target. 

Dodson’s next consideration in model development 
was the synthesis of the individual cost estimating 
relationships into the overall cost model. He 
recognized they must cover all costs of interest. 
Also, costs were defined so that double-counting was 
avoided. He also defined all units in dollars of the 
Same purchasing power. 

Dodson based the Work Breakdown Structure for this 
study on the life-cycle elements of the Demonstration 


and Validation, Full Scale Development, Production, and 
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Operations and Support acquisition phases. The first 
two phases separated recurring and nonrecurring costs. 
Recurring costs represented those costs directly 
associated with ere fabrication of prototypes. 
Nonrecurring costs included all other costs, primarily 
erermneering design and component testing costs. 

All cost relationships were presented in thousands 
of 1976 dollars. Provisions for translating those cost 
figures into specified future year constant dollars 
Were included. 

The cost elements at the bottom levels of the Work 
Breakdown Structure included the hardware associated 
with the laser systems with other cost elements such as 
system test and evaluation and project management. 

Computer subroutines developed by Dodson enabled 
the model to carry out types of calculations common to 
a number of high-energy laser subsystems. The required 
inputs to the high-energy laser systems cost model 
mre Luded ; 


es Choice of laser type, like chemical or gas 


dynamic. 
2. Device output power in watts. 
3. specific power in kilojoules per pound. 
4. Number of laser shots per mission load. 
5. Shot time in seconds per pulse. 
On, Pulse recurrence frequency in pulses per second. 
7. Electrical efficiency, consisting of device 
output power divided by prime electric power. 
8. Output beam diameter in centimeters. 
9. Number of turrets per aircraft. 
10. Number of adaptive optics actuators per mirror. 


11. Number of flight hours. 
The model also provided optional inputs to the 


user. The user could choose among different procedures 
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for calculating fluids costs, avionics costs, or 
operations and support costs. For example, for fluids 
the user provided the flow rate for each fluid, which 
was then combined with cost and lasing time information 
to establish total flutde casita. If flow rates were not 
available, the model could assimilate the values of 
specific power and lasing time with internally-stored 
information about the proportions of individual fluids 
and their costs, to determine total fluid costs. 
Tables with the numerous’ characteristics for fluids 
were provided in the model. Fluids represented a 
variety of individual reactants and diluents used in 
high energy laser systems. The primary characteristics 
contained were cost-per-pound figures. 

The High-Energy Laser Systems Cost Model provided 
the following outputs: 

1. Listing of user provided system inputs. 

2. List of Avionics inputs as selected by the user. 

ai Intermediate set of results for fluids, including 
calculated values of weights and costs for 
individual fluids. 

4. Printout of fluid-usage parameters for the 
individual fluids selected for the case under 
study. 

5. Listing of the Operating and Support cost 
parameters used in calculating these costs. 

6. Detailed presentation of life-cycle costs by cost 
element in the Work Breakdown Structure. 

Costs were presented in thousands of constant- 
dollars based on 1976 as the base year. Successive 
levels of aggregation were shown in the cost model. The 
model also summarized by cost element the individual 


cost estimating equations utilized. 
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The following example highlights Dodson’s model. 
The validation phase, one of the four life-cycle 
phases, consisted of nonrecurring and recurring costs. 
Nonrecurring costs were the sum of these elements: 


ie Airborne system costs, which included: 
a. Device costs. 
Ds Power supply costs. 


c. Fluid supply costs. 


d. Optics, pointing, and tracking costs. 
e. Avionics costs. 
Cae Aircraft modifications costs. 


System test and evaluation costs. 
4. Project management costs. 


The device cost for airborne systems was the sum of 


individual costs developed for the device, 
diffuser/ejector, and associated instrumentation 
controls, along with auxiliary power elements. The 


mathematical cost estimating relationship generated for 
chemical laser devices was cost equalled 15.309 times 
device output power (watts). Device output power was 
first scaled to the .44 power. 

Dodson’s model did not express separately SOA 
extension costs. In the next section, the Air Force 
Spacecraft model is discussed, with particular emphasis 


on the use of engineering design complexity. 


D. UNMANNED SPACECRAFT COST MODEL 

The 1981 edition of the Space Division Unmanned 
Spacecraft Cost Model is considered by the aerospace 
industry estimating community as the most widely 
applied spacecraft cost estimating tool. [Ref. 25] 
The model’s purpose is to collect historical 
cost data for use in a parametric cost estimating 


relationship framework. The model is used to formulate 
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more responsive cost estimates for long range planning 
studies and future spacecraft systems. 


The model is organized into a data matrix. The 
matrix ies subdivided into areas Of ACL V iaeye 
subdivisions of work, and elements of cost. Areas of 
activity are specific or general hardware 


classifications, such as aerospace ground equipment. 


Subdivisions of work indicate processes associated with 


more than one hardware item, such as design or 
fabrication. Elements of cost represent the standard 
cost accounting categories of engineering labor, 


material costs, manufacturing overhead, and general and 
administrative expenses. 

The subdivisions of work are identified as 
nonrecurring and recurring costs. A time-phased method 
determines the break between the two types of effort. 
Nonrecurring costs begin with concept development and 
stop when the qualification test of the prototype is 
complete. 

Three examples of the areas of activity are the 
platform, electrical power supply,and the attitude 
control. The platform is part of the spacecraft 


structure, and bears the majority of spacecraft dynamic 


stress loads. Examples of cost drivers for the 
platform are. “structure Vexsnt volume, and mass 
density. The electrical power supply generates, 


converts, and distributes all electrical energy between 
spacecraft components. Examples of cost drivers are 
electrical power supply weight, battery weight, total 
vacuum impulse, and action time. 


ire attitude control system maintains the 
spacecraft in the required orbit. The system can be 
stratified into eee design categories or _ two 
functional categories of equipment. Cost drivers are 
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oligay, 


weight, wet weight (with fuel), total impulse, 


ieee ting lite» angular drift, and altitude. 


The ground rules for the cost model are: 


The model addresses only unmanned earth-orbiting 
spaececrart. 

Cost estimating relationships are obtained by 
relating costs at the subsystem level to physical 
and performance characteristics. 

All cost estimating relationships are based upon 
burdened costs, so the model consists of the 
EQtal cost through general and administrative 
expense cost estimating relationships. 
Penimety—-tivesspercent average learning curve is 
Meecd toe@erive unit costs. 

The cost estimate is expressed in 1979 constant 


dollars. 


Based on the ground rules, starting point cost 


estimating relationships are generated. For the three 


abeaS 


of activities considered earlier, the derived 


mem@itession equations were: 


thea 


Platform costs = 7414.46 + 22.6 X, with X 
representing platform dry weight in pounds. 
Electrical Power Supply costs = 360.97 + .0165 X, 
with X representing the product of electrical 
power supply weight and beginning of life power 
iit bS.. 

Retr tude Control costs for the attitude and 
reaction control subsystem = 426.49 + 31.47 X, 
with X representing the dry weight of the 


attitude and reaction control. 


The cost estimating relationships listed above were 


derived after examining all program cost data on 


scatter diagrams. Next, regression analysis was 


performed for several parameters. Further analysis was 
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performed on the cost drivers” =which  sienitteamely 
int luenced cost. Transformations were performed on 
selected variables via multiplication, square roots, or 
logarithms to create synthetic variables in an effort 
to find the most influential cost drivers. Finally, 
the data was' stratified for all data points’ to 
determine homogeneity of the points. 

Once the cost estimating relationships were 
generated, They Swere normalized to account for 
inflation, influences of alternate design concepts and 
new technological breakthroughs. In the normalization 
process, actual cost data are evaluated with respect to 
quantifiable subjective parameters. The parameters 
enable the actual cost data to be adjusted to a common 
base at the subsystem level. Two subjective parameters 
selected were technology carryover and complexity of 
design. 

The technology carryover cost factor measures the 
state-of-the-art of technology at different periods of 
time. The technology carryover measurement scale is 
divided into five levels to capture the degree of 


engineering learning over time. The five levels are: 

Le 1.00: the item is substantially beyond the SOA. 

Oe .75: the item is slightly beyond the SOA. 

on moO ; the item is within the SOA but no 
commercial counterpart exists. 

4. cua the item will involve a minor modification 
of commercial items. 

Ge -10: the item will be -procured off-the-shelf. 


Programs can be examined by the Unmanned Spacecraft 
Cost Model by the complexity of design cost factor 
also. The first step is to identify subsystem 
operational criteria which could reYawe cost to the 


aGéepree of Veomnplexi ty. Descriptors must be chosen to 
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realistically assess the operational criteria. Each 
operational criterion is ranked against a base value of 
100 percent. The evaluation is compared to the 
subsystem complexity factor matrix. The relative 
ranking indicates the degree to which the operational 
criteria affects the costs of developing the subsystem. 

To obtain the normalization cost factors, a 
comprehensive study of industry experts was conducted. 
A weighting scheme was devised to generate one 
mormalizZzatYon factor for each subsystem from the 
composite of technology carryover, complexity of 
westen, and inflation factors. Each subsystem’s raw 


cost data points from the initial cost estimating 


relationships were divided by the composite 
normalization factor to yield a set of normalized cost 
data points. The normalized cost will always’ be less 


tiam the initial point design cost estimate. 

The normalized cost estimating relationships 
enable the cost analyst to perform trade-off studies 
for near-term conceptual programs. Hey Seeer mit 
calculation of costs for more specific spacecraft 
programs. The Unmanned Spacecraft Cost Model via an 
appendix provides a summary of all the normalized cost 
estimating relationships. 

The next section outlines the basic concepts behind 
the Freiman Analysis of Systems Technique (FAST), 
developed by Frank Freiman. [Ref.26] 


E. FREIMAN ANALYSIS OF SYSTEMS TECHNIQUE (FAST) 
Freiman developed iy @ Us FAST parametric cost 

estimating system to evaluate the cost impact of 

variations in schedule or design. Freiman’s system 


differs from the conventional parametric models, such 
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as the Unmanned Spacecraft Cost Model, in the following 
ways: 

Le It quantifies technological phenomena underlying 
design which cause costs to vary with size and 
design. 

Oe It reduces need for lengthy historical design 
versus cost records. 

Ons It allows synthesized data points to be used for 
initial development estimates. 

The FAST methodology involves analysis of the 
fundamental concepts behind technology variance with 
cost per pound. Freiman found advanced SOA 
technologies provide more energy per unit of design 
mass than those within the SOA. The same performance 
for SOA extensions can be accomplished with less 
equipment mass. Freiman’s theory is that to advance a 
design mass, more energy per pound must be utiliZed. 

The FAST model estimates costs by its class of 


technology. Its seven basic types, in hierar cheval 
order, are electronic, electrical, heat, motion, 
mechanical control, containment, and support. Freiman 


also used the weighted average level of the technology 
with the degree of performance desired to categorize 
equipment types by design mass components. His five 
basic types of design masses are: 
1. Energy Conversion Mass - converts one energy form 
to another. 
2. Design Overhead Mass - added due to 
inefficiencies of design. 
Application Mass - required to transfer energy. 
4. Dimensional Mass - required for physical coverage 
of a system. 
5. Conditional Mass - required for environmental or 


personnel safety reasons. 
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Based upon the design mass_ and technology type, 
FAST can quantify cost per pound of the electronic 
system being estimated. Via computerized mathematical 
equations the cost estimating relationships among cost, 
weight, and technological complexity can be expressed. 


FAST also simulates’ the thought processes) of 


successful managers. FAST’s methodology is derived 
from the way managers intuitively assess the cost of 
SOA advancements. FAST is designed to simulate 


behavioral responses ranging from establishing the data 
base to exercising "what-if" capabilities. 

FAST systems feature stheefol lowing: 

1. Accept tailored inputs from varied design and 
manufacturing circumstances. 

ae Project funding requirements via graphic and 
alpha-numeric displays. 

ue Provide risk measurement through display of cost 
uncertainties for each cost segment. 

4. List detailed complexity values for commercial 
and industrial items. 

5. Are user-friendly, unlike the more specific cost 
models such as the ground surveillance radar cost 
moe i. 

FAST is useful as a check for conceptual stage cost 
estimates, although it must be calibrated for each 
im@ividual user. Tables 1 and 2 represent sample 
outputs for the FAST cost estimating model for 
electronic equipment. Fable Seiamdivdded into five 
primary sections, broken down as follows: 

1. Total estimated costs of the line item, with 
subtotals for engineering, production, and 
installation costs. If necessary, costs for 
schedule delay could be included. 

oe Cost uncertainty distributions, for risk 
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evaluation. Three confidence tuverval =) anes 

from seventy to ninety percent are shown for the 

engineering, production, and installation 
subtotals. Note the total assumes the covariance 
equals one. } 

Characteristics of the equipment’s energy 

sources. The primary energy required is 

expressed in kilowatts, while secondary items 
such as pressure and temperature are also 
summarized. A synthetic FAST energy variable is 
calculated and shown below the primary and 
secondary characteristics. 

Production cost data, using average unit costs. 

Figures for the total estimated production costs, 

manufacturing costs, and the theoretical forecast 

are shown. In this example manufacturing costs 
are the same as production costs, due to lack of 
beginning work in process inventory. 

Input data. The factors actually entered by the 

cost estimator are presented. Among the inputs 

are; 

a. PRJGLOB: Overall project global inputs are 
shown. This line requires an escalation 
control factor for inflation, the year of 
economics to be used, learning curve factor, 
and a cost multiplier index. 

b. FILE and FORMAT input different types of 
system options. 

c. GLOB = Further global inputs are included, 
specifically the platform specification 
level, engineering design mass type, year of 
technology to be used in production, and a 
weighted energy value based on kilowatts, 
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british thermal ULEvs,e Pressure, and 
temperature. 

d. WTVOL: Weight and volume factors, along with 
their weighted values, are shown. 

e. MXLINE: Manufacturing complexity values for 
the production of the line item are given. 

fc PCOST;, ECOST @arcost- Inputs for basic costs 
of production, engineering and installation, 
along with quantities and the year of 
economics, are shown. 

Deel oc. BPSoCHD, ISCHD : Values for start and 
completion dates, complexity values, and 
skill levels are presented for production, 
engineering, and installation schedule 
factors. 

eb coder cone (LOH: Production, engineering, 
and installation material, manufacturing 
overhead, labor, and indirect labor factors, 
along with an aggregate labor rate for each 
phase, is presented. 

Table 2 shows the cost distribution by total labor 
hours for engineering, production, and installation. 
Also, a production profile graph compares’7~ the 
cumulative funding for the project against the 


estimated cumulative expenditures, by quarters from 
1984 to 1991. The expenditures indicate actual 
disbursement of manufacturing costs, while funding 


indicate the budgeted figures. 


F. SUMMARY 
This section highlighted features of four different 
cost estimation models. Chapter Four presents the case 


study description of Litton Applied Technology’s cost 
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estimating process for the AN/ALR-67 Radar Warning 
System. 


60 


— 


PROCEC: 
LOCATiS 


re. 
ocr s 


ENG 

PRCO 

oe 
aoe 


PCOS? 


PSCHD 


PLOH 


Ecos 


ESCHSD 


ELSH 


2COST 


2SCHD 


20H 


@eeee FAST-E eeeen 
© TEST UNIT DATE: MON, JUL 14 1986 09:85.23 
N; GERMAISTORN , MD FILENAME: FASTE.EXAMPLE 
OETAIL ECONFILE: 
eeeee DETAIL ITEM eeeee 
SUNITS -« 1,000 BASIC COST SCHD PENALTY ROTAL COST 
ENGINEERING T37<.d 0.00 797229 
PRODUCTION 1,301.45 0.00 1,301.45 
aNSTALLATION 519.84 0.00 519.84 
TOTAL ACQ. 2,618.48 0.00 2,618.48 
eeeee COST UNCERTAINTY OISTRIBUTION eecece 
“FROMe 70% -TO- “FROM= 380% <-TO- =) RON ow gOS = 10 
723 871 6380 914 619 97§ 
Deas LoS 1,184 1,419 Pe 1,48¢ 
476 564 450 590 413 62€ 
2,427 2,810 2,414 2,920 See 3,08< 
eevee CHARAGCRER Sc. 20S eeeee 
PRIMARY VARIABLE QVANTIZ 
KILLWATT (PKW ) 250.000 
SERCNCARY 
SReawon, siaeenaa (S55. a s,rece. 900 
PRESSURE (PST) CSFRESS) 70.000 
TEMPERATY (Fs) (o2G Ee) 125.000 
FASTE EQUIVALENT 
PASSE oot. 2.6% 
TENVLECFASTE FAS. C. 2588 
@eeee PROOVCTION COST CATA #eeee 
DEnAGE ware CoS. 9 CASE Ch: 
BOeAe PROCUCI ION COST MAMVFACTURING CCST THECRETICAL FPCOST 
26,029.93 26,029.03 40,466.78 
@eene TNPUT SATA *e0eee 
ESCAL GECON FPER SGU Ee era 
8 2.900 1383 oc 1.999 wee 
ae Ser Dia ee RERUN REMBOX 
D Q Q 2 
ALA Cosa UNCERT LWSRMAT INPUT FGORAFH Foes 
1 0 0 0 Q 0 2 
Pier ENTYPE MATVAL TYEAR Toe AL 
1.300 70.900 2-00 1988 BW eee PRS. cores Ju eu: 
WT WIFAC VOL VOLFAC 
1,060.70¢ 350.0 24.15°¢ 400.00 
PMX MAat re ELWT ELMX STWT See 
0.200 4.2808 0.0 0.000 0.0 0.09 
FGOsT PQTY PECON FOAK 
1,201.45" 5Q. oes 1.000 
PSTART Pron PSCHOX PROMX PYEARC FOloC 
OEC.1989 NOV.1991* 100.00 Jia 7 Ze NONE 
PMATL PMOH PLABOR PLOH PLRATE 
0.30 0.00 0 Jai 1.06 Live 2 
ECOS? EQTyY EECON EMX ENEW 
797,19 ek 2 ae 0.850 gi. 700 
ESTART EFIN ESCHDX LEVEL EYEARC EO. oC 
JUN, 1988 OcT.3989 i243 0.700 ,3e9" NONE 
EMAL & EMOH ELABOR ELOH Beka te 
On 5 0.00 0.26° 1.48¢ Zao 
Eas 7 IGar IECON IMX 
519.84¢ 50. 1989 1.250 
ISTART LEIN ISCHDX TCEVED IYEARC eS 
JUL.1990 FES. 1990" 80.00 1.000 is83¢ NONE 
IMATS 2MOH ILABOR TLOK ILRATE 
0.15 0.00 0.46¢ 0.85¢ 13.40 
CONTIN ACO 
Q 0 


O> 
ES 


LEE a2 


@enne COST DISTRIBUTION seen 


an ENGINEERING PRODUCTION INSTALLATION TOTAL 
(HRS) -( 9, 2283575568 )> 4 Ne a On 27,868.99) ( 64,725 0275 
COST 209.34 441.48 239 4 
e « *% ° 4 890. Zo 
Cie 308.83 469.54 202.42 980.79 
§18.18 911.2 441.86 1, 871506 
MATL 
aos 279.02 390.44 77.98 747.43 
an ae 0. 0:60 6.00 ORE Jello) 
279.02 390.44 77.98 747.43 
TOT 797.19 14 302.405 $19.84 2,618.48 
aennen FAST<-E 2 a0ean 
“PROJEGL: TEST UNI DATE: MON, JUL 14 1986 09:59:58 
LOCATION: GERMANTOWN, MD FILENAME: FASTE.£XAMPLE 
TEM: bDEwALL ECGCONFE ILE: 
eeene DETAIL ITEM seeee 
@enne PRODUCTION PROFILE *#e08 
89 90 91 92 93 94 
re ee [ooeee | eceee Jonen- lenne [7--= [===e=| -- ~~~ Jeoneny 
C 15 -Db #= basic exp. = 
O '@ @ est. fund , 
S$ | 
5 | eeeereebbbobbp 
vores a ane bb eee - - aa 
| “—_- ~ 
N | * 5 a 
Be) 2 
T 9 - * b aoe = * | 2 
H amawenanetane pe 2e | 
0 e db e fe f 
U | ‘ * | 
S 6 = ® « b 2 « ete - = 4 
A | e@ b e @ 
N | * b ; ee ) 
5 e e fe | 
5 ae e+ »b «+ ee > “i 
® * d ® #e 
l a dD 2 ae 
5 e b o ff 
2) 0 oy eer ee | ewe te peworoen | enw tea! 22 22 @ (eee BR > eweeewnex  eeee2e .ieenaee:ieaeeneeae= «+ G 
89 90 9. a 93 94 
YEARS 


(1 MOM PER Dee EME 


a7 


TABL 


7 


ee ot 2) 


eeees FSTIMATED PROJECT PROFILE «anes 


(EOS tee a ROC SANCS, 


Coan TEAR CUM. ie Noo Cee EXPENOG TURES 


Cer 1389 oor orAS 
JAN 320 Be. OS 66.4 
APR L339 S1l6..0 32 ie 
JUL 2320 Sl6. 3 62ee> 
Cet 193:0 pe a oS S99 
JAN oe a 29 Do ie 09 Sete 
APR 2 oe. Leo eS 1) 222.4 
JO Loot ee oa. 5 i 27 SS 
Ces 1392 On D ae 3 9 eo 
NOV igo. dO: lane ao) Olen 


aanen ESTIMAZLEO FUNDINGS 80888 
(GOS. BY CHOLSANDS) 


MONT! YEAR FUNOINGS 
OCT 69 816.3 
Get a 20 wo 
OcT oat 6.1 


SISO PROOUCTION QUANTITY BY FISCAL YEAR ***** 


YEAR ENDING 


MONTH YEAR FISGAL QTY 
Ser 1990 1 
Ser Tes oeL 48 
SEP 1992 1 
TCTAL 50 


QO) 
tad 


LITTON APPLIED TECaNOLOG IS ecus . 


IV. DES@RI L 
ES ee EN 


As INTRODUCTION 


This chapter describes Litton Applied Technology’s 
cost estimating system for development projects which 
extend technology beyond the SOA. This case study will 
primarily focus’ on the AN/ALR-67 Radar Warning 
Receiver, also called the Advanced Countermeasures 
Control and Warning System. The chapter is divided 
into the following sections: 

a Litton’s background in development of radar 
warning receivers. 

2. Description of key physical and performance 
characteristics of the AN/ALR-67 Radar Warning 
Receiver. 

3. Litton’s pricing and estimation system. 

4. Cost estimation methodology for derivation of 
OTrec te eoe tar, 

5. Delineation of actual costs incurred by Litton 
from development through initial production for 
the AN/ALR-67 Radar Warning Receiver. 


B. BACKGROUND FOR RADAR WARNING RECEIVERS 
di. General Background 

Litton Applied Technology is a division of 
Litton’s Electronic Warfare Systems Group. Currently 
located in south San Jose, California, Litton Applied 
Technology employs approximately 1800 people in the 
research, design, manufacture, and support of defense 
electronics systems. Besides radar warning receivers, 
they also specialize in integrated electronic warfare 
systems, space and strategic defense systems, and 


flight training and operational simulation systems. 
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Applied Technology actually was an independent company 
until 1983, when it became a division of Litton. 
2. History of Radar Warning Receivers 

Applied Technology first became involved with 
radar warning receivers in 1965. The concept of radar 
warning receivers was initiated by a special Air Force 
task force commissioned to develop methods to counter 
Soviet surface-to-air missiles. In 1965, North Vietnam 
developed a complex air defense system closely 
coordinated through the use of communications) and 
radar, based on Soviet technology. The Air Force’s 
research committee reviewed new concepts for warning 
and jamming equipment which could counter’ the North 
Vietnamese air defense threat. In November 1965, 
Applied Technology received a cae mil lions dollar 
production contract for 500 AN/APR-25 Radar Warning 
Receivers from the USAF Sacramento Air Material Area. 
The contract was Applied Technology’s first major 
production contract, since previous defense experience 
was limited to technology applications for 
intelligence programs, where large production runs 
consisted of only ten units. 

The design philosophy for the AN/APR-25 
centered on gathering as much signal information as 
possible. Crystal-video detection techniques were used 
in the threat bands to determine the hostile 
equipment’s relative direction. 

In the late 1960’s Applied Technology updated 
the AN/APR-25 with the AN/APR-35 Radar Warning 
Receiver. New technology such as automatic time/video 
correlation circuits and a new superheterodyne analysis 
receiver were added. Also, improvements in operator 
interfaces were added to the equipment to enable the 


Electronic Warfare Orricer to instantaneously 
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communicate to the pilot which directions surface-to- 
air missiles were fired from. 

The next generation of radar warning receivers 
was introduced in 1970, based on the new Soviet threat 
to the Mediterranean region. The AN/ALR-45 became the 
fair st digital system to incorporate hybrid 
microcircuits using digital logic and clock drivers. 
This generation’s design philosophy emphasized the need 
for sorting analysis of emitter types and 
prioritization of lethal pulses. Non-lethal threat 
information could now be discarded. 

In 1971 Applied Technology, in a fundamental 
change, became deeply involved in computer design 
evolving from analog circuit design. In 1972 the 
company developed the Applied Technology Advanced 
Computer (ATAC) specifically for electronic warfare. 
This computer was capable of being preprogrammed at a 
squadron level. The ATAC computer’s volume was 96 
cubic inches, with power consumption 45 watts, and an 
input/output rate of 1.25 megawords per second. 

In 1975, the ATAC computer was used in the 
development of the AN/ALR-67 threat warning program. 
Integrated power management systems now collect and 
analyze multiple threats to enable optimum jamming. 
The AN/ALR-67 was developed due to the Navy’s need for 


increased speed and prioritized threat warning 
information. The AN/ALR-67 is now deployed on the F/A- 
18A, CF-18, F-14A, F-14B, A-6E, A-6OF, AND AV-8B 


alsmeraf t% 

The next generation of radar warning receivers 
to be fully deployed will be the AN/ALR-74 threat 
warning systems, as of 1987. The AN/ALR-67 is 
completing the full-scale development phase. 
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C. PHYSICAL AND PERFORMANCE CHARACTERISTICS 

The AN/ALR-67 Radar Warning Receiver’s primary 
function is to notify the radar operator of the 
presence of threatening signals. The AN/ALR-67 is 
designed to inform the pilot of how many hostile radar 
Syocems amem@active imehis flight area. The AN/ALR-67 
identifies the type and relative location of each 
threatening radar signal. 

The most important performance criterion of the 
AN/ALR-67 is response speed. The AN/ALR-67 is designed 


to allow the pilot time for evasive action against 


potential threats. Also, the radar warning receiver’s 
response time is quick enough to provide accurate 
relative direction information on hostile threats 


immediately following aircraft maneuvers. 

To be effective, the AN/ALR-67 must transmit the 
critical parameters of the threat to the aircraft 
jamming system in digital form. The effectiveness of 
the jammer’s electronic countermeasures depends on the 
AN/ALR-67’s receiver acquisition time. 

The second performance criterion for the AN/ALR-67 
is threat identification. The radar warning receiver 
characterizes hostile signals by their modulation 
characteristics and range of RF frequency operation. 
RF frequency pulse trains often occur simultaneously, 
so the AN/ALR-67 must be capable of unambiguous signal 
identification. 

The AN/ALR-67 is also designed to look at selective 
frequency bands so high duty signals can be analyzed 
independently without interference. The pilot receives 
information from the AN/ALR-67 Radar Warning Receiver 
by visual displays on a three-inch diameter cathode ray 
tube in the cockpit. The pilot receives data as 
following from the AN/ALR-67. 
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1. Relative direction of each signal to an accuracy 
of 15 degrees to 30 degrees. 

ae Indication of each signal’s strength. 

oF Identification by symbols of specific radar 


types. 
The design of the AN/ALR-67 Radar Warning Receiver 
is constrained by three practical factors: volume, 
power consumption, and cost. The allocated volume for 


the radar warning receivers has remained constant since 
1eOiGid . The increase in customer requirements and the 
continued development of electronics microwave 
techniques has led to high package density, which 
imposes the constraintS on power consumption. Cost 
constraints must be met through system tradeoffs. A 
six decibel difference in receiver sensitivity 
requirements could mean the difference between 
development of crystal video receivers, or more 
expensive wideband superheterodyne receivers. 

Processing of signals through the ATAC computer has 
replaced human interpretation of audio and visual 
information. The AN/ALR-67 has a_ tangential receiver 
sensitivity between -50 dBm and -60 dBm, at a 10 -MHZ 
video bandwidth. System sensitivity experiences losses 
due to cabling and filtering. 

Search speed limitations place another constraint 
on the AN/ALR-67. The ratio of the RF frequency search 
band to the receiver instantaneous bandwidth determines 
the length of time it takes a receiver to intercept a 
potential signal. 

The most recent advances in microprocessor 
technology enable radar warning receivers to have each 
element of the distributed receiver system to be 
controlled independently by a computer. The AN/ALR-67 
was the first radar warning receiver to control its 


system operations via computer software. 
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The individual subsystem components of the AN/ALR- 

67 Radar Warning System are the: 

Computer. 

AZimuth indicator. 

Cencrol indieator. 

Quadrant receiver. 

Special Beceiver. 

Receiver antenna. 


NI Oo OP W DY KF 


Quadrant antenna. 


Their size and weight characteristics are shown in 
able 3. 
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De LL PSON APPLIED TECHNOLOGY’S COST ESTIMATING SYSTEM 

Litton Applied Technology has developed a complex 
matrix structure to manage its proposal estimating 
system for government contracts such as’' the AN/ALR-67 
Radar Warning Receiver. Its approach is designed to 
accommodate government procurement regulations, 
compliance with the Cost Accounting Standards, specific 
customer requirements, and its own company policies. 
The stated goal of their cost estimating system is to 
be systematic and consistent. 

Litton’s Proposal Pricing and Estimating Manual 
defines its five primary tasks as: 

1. Define requirements in amanner which allows 
specific work elements to be performed. 

2. Develop a work breakdown structure and cost 
matrix compatible with the cost collection 
system of the cost elements. The elements must 
be measurable and definable through task 
descriptions which are consistent with the 
statement of work requirements. 

oi Identify and develop significant milestones and 
schedules for each work element and a realistic 
program schedule. 

4. Prepare data to serve as the basis for the review 
of all cost estimates. 

5. Review and present the related cost experience, 
historical data, and detailed cost estimates. 

The Vice President of Business and Financial 
Operations has’7 overall responsibility for Litton 
Applied Technology’s cost estimation system. The two 
primary subordinates who assist him are the Director of 
Proposal Cost Estimating and Analysis and the Manager 
of Proposal Operations. 

The Director of Proposal Cost Estimating and 
Analysis is primarily responsible for the development 
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of cost proposals for division products. His “division 
must establish and maintain effective cost estimating 
tools and techniques. He also ensures cost proposals 
comply with customers’ cost proposal requirements and 
regulations. The division must support internal cost 
reviews and establish the budget baselines for the 
project cost control. The direct planning, monitoring, 
and prioritization of all cost proposal activities are 
conducted under his cognizance. 

Litton Applied Technology’s corporate guidelines 
hold the Director 7 aecos se Estimating and Analysis 
directly accountable for: 

1. Formulation of guidelines for all proposal 
pricing preparation and reviews in accordance 
with division policy. 


2. Development of overall division cost proposal 
plans. He must define an schedule the 
prerequisite support from line management. The 


plans must cover the statement of work, basic 
assumptions, cost data, and problem 
identification and resolution. The conducting of 
reviews with line management on major cost 
proposals is included within this task. 

3. Support of cost proposals during the customer’s 
evaluation cycle. He must develop and establish 
negotiation cost positions. He also assigns and 
approves the members of the negotiating team. 

The director ensures cost updates and cost 
disclosures are in accordance with customer 
regulations. 

4. Review current division cost performance trends 
to ensure that such performance is considered in 
all proposals. 

Divs Direct resources dedicated to the development and 


maintenance of the pricing data base. He is also 
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accountable for implementing advanced estimating 
techniques and systems for the division’s 
proposals. 

6. Support design-to-cost analysis projects of the 
division’ 

The Manager of Proposal Operations is accountable 
for managing Litton Applied Technology’s capability to 
Gespemed Go cilstomers requestsw for cost estimates. He 
caamres thes diwision cost proposals are complete, 
accurate, and minimize cost risk. He serves asa key 
interface with the functional line departments, program 
office, senior management, auditors, and customers. 
Finally, he must review and approve all formal cost and 
price proposals for content and conformance with 
government regulations, public law, and customer 
reogUdaremnents. 

The cost proposal cycle for Litton Applied 
Technology consists of thirteen distinct steps. The 
Bidividual steps, in successive order, are: 

iteereceipt and acceptance of the customer’s request 
ELOreproposa:).. 

2. Assign a proposal team manager. 

Issue the proposal authorization to proceed 
order . 

4. Assemble the proposal team, with representatives 
from Engineering, Quality Assurance, Operations, 
Contract Administration, and Proposal Cost 
Estimating and Analysis. 

2. Conduct proposal team planning meetings. The 
team must establish a tentative program schedule 
and develop the work breakdown structure. The 
bid matrix, which designates the individuals 
responsable for the cose™estimate for every 
single element, is promulgated. 


6. Brief senior management of the team’s plans at a 
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one-hour kickoff meeting. 
fe Review the proposal strategy and technical 
approach at previously defined intervals. 
Se Identify all material requirements. 


oie Collect and analyze the cost estimates. 
10. Conduct final management review. 
Midlives Prepare the proposal submittal volume. 


12. Submit the proposal to the customer. 
HN Rs Complete audits and negotiations by the customer. 
The Key member ins this matrix (concept Wserescose 
estimating is the proposal analyst assigned from the 
Proposal Cost Estimating and Analysis Division. He is 
responsible for the compilation and analysis of the 
estimated costs submitted by the personnel designated 
in the bid-matrix structure. Also, he performs "make- 
or-buy-analysis" on selected high dollar value parts. 
The general cost estimating approach by the 


proposal team is the "bottoms-up" cost engineering 
method. The bottoms-up method is utilized for every 
proposal, from development of new designs to full-scale 
production cost proposals. The key steps’ the team 


performs for cost analysis are: 

dls For existing design configurations, the proposal 
analyst retrieves a computerized bill of material 
from the on-line material pricing system. 

2. For new designs, the responsible functional 
organization will generate a bill of material and 
send it to the proposal analyst. 

3. The proposal analyst develops a priced bill of 
material with support from the Procurement 
Material Pricing Department for inclusion in the 
cost proposal. 

4. The functional organizations submit their direct 
and indirect labor estimates to the proposal 


manager for review. 
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9. All functional cost estimates are given to the 


Proposal analyst to create the preliminary cost 


ole: . 

Or: Preliminary cost reviews with senior management 
are conducted. 

7. The proposal analyst receives the cost review 
results. He next compares them with the cost 


history on similar programs for reasonableness. 
8. All team members review the revised cost 
estimates to ensure they are factual, verifiable, 
complete, and support the proposed amounts. 
9. The proposal manager and proposal analyst brief 
the company executives at the Final Management 
View to defend their cost estimates. 

For every stage in the cost proposal development 
process, checkoff sheets are generated. Signatures by 
the responsible individuals are obtained to indicate 
completion for every step. The proposal analyst 
primarily verifies the functional groups’ cost 
estimates for labor hours’) and material quantity. On 
Production contract proposals with little risk, the 
proposal analyst generates the cost estimate himself 
based strictly on historical data. Company officials 
estimate that an average of three complete cost 


estimate reviews are conducted before final approval. 


Ee. COST ESTIMATION METHODOLOGY FOR DERIVATION OF 
DIRECT COSTS 
Litton Applied Technology’s Cost Estimation and 
Pricing Manual defines direct material as the cost of 
material used in making a product which is directly 
associated with a change in the product. Litton’s 
direct material base is comprised of raw materials, 


purchased parts, and subcontracted items. For raw 
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materials, a raw stock factor is applied to the total 
estimated material base to account for these costs. In 
their system raw materials which require further 
processing are treated as indirect costs. 

When budgetary estimates, also called ROM (rough 
order of magnitude) estimates, are requested for 
supplied of existing products, they are based on the 
most recent firm estimates of the same or similar 
items. The prior estimate is adjusted by the analyst 
for quantity differences and the degree of complexity. 
An annual escalation factor is added to the prior 
material estimate to account for period of performance 
differences. The difference in quantity is adjusted by 
utilization of a ninety-five percent improvement curve. 
The proposal manager, assisted by the functional 
engineering team members, provide the complexity factor 
for material estimates to tne cCostweanatycc. 

Follow-on spare parts estimating is based on Litton 
Applied Technology’s on-line computerized material 


pricing system. The source data for the computerized 
system represents the most current configuration for 
released part lists. Proposal analysts have the 


capability to extract the purchase order history of 
selected spare parts dating back to 1975. The analysts 
incorporate this information to the priced bill of 
material. The computerized on-line system can generate 
a priced bill of material by either the individual 
assembly or part number, or a consolidated group of 
part numbers. The computer’s primary files from which 
the bills of material are generated are the engineering 
configuration file and the manufacturing configuration 
file. 

The Proposal Material Audit Report is Litton’s name 
for the computer generated list of material prices 


based on the purchase order history. Litton s) policy 
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for high dollar value/high usage parts dictates a 
Peeterence for current vendor quotes instead ~of sole 
reliance on previous history. Their normal procedures 


mandate at least three competitive quotes’ must be 


obtaimed. Follow-on spare parts are also adjusted in 
quantity by Litton’s ninety-five percent improvement 
Caaiay Cx. Escalation factors for the period of 


performance differences also are included. 

If there is no previous history of a material 
purchase, the proposal analyst requests via a standard 
material pricing request form that the Purchasing 
Material Pricing Department obtain vendor quotes. The 
functional proposal team members’) provide the physical 
description to the proposal analyst. 


Estimates of material requirements for the SOA 


development pea jects, such as the conceptual 
exploration phase of the AN/ALR-67 Radar Warning 
ays bem , are based on the "bottoms-up"”" system 
engineering approach. Thee coscmeZanteengsineer sat the 


lowest level of the work breakdown structure develops 
Ene bill of materials after*detaMie@@™ analysis of the 
pmOpOeSed sesd#ens configuration specifications. The 
responsible engineer provides “the )~quanti tye part 
number, and description of the required materials in 
accordance with the bid task matrix instructions. The 
required bill of material is passed to the proposal 
analyst via the Material Cost Estimate Detail Form. 
The material requirements for new developments are 
based on: 
PtP larity to existing equ®pment. 
2. Vendor catalog items. 
The priced bill of material based on "similar to" 
equipment is adjusted by a qualitative complex faetor . 


For items never previously purchased, the complexity 
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factor is determined by a consensus among members of 
the proposal team. 


Once the dollar estimates FoR the material 
estimates are gathered, the proposal analyst next 
develops the Principle Items List. This @btstewil 
include a previously determined numerical sampling of 
high dollar/high usage items. The written rationale 
for each item must include: 

1... Part number. 

2. Known or anticipated source. 

3.0 Tota lg@auanimsty . 

4. Unit and total price. 

Dis Competition status. 

6. Basis for establishing the source. 

7. Determination for reasonableness of cost. 


The proposal analyst justifies cost reasonableness 
for the selected items based on inputs from the 
Procurement Material Pricing Department. The proposal 
analyst next presents the list for approval to the 
proposal and business area managers during formal cost 
input coordination meetings. 


The Direct Labor Narrative Statement is another 


document required for the cost estimate of the 
proposal. Upon completion, both the functional 
proposal team representative and the cognizant 
functional director are required comes on ic. aie 


document should show enough detail to separate labor 
for each distinct operation. Each Operation is 
identified by an engineering cost center and labor 
category. The estimates for direct labor costs must 
consider whether prior relatable efforts exist or if no 
verifiable labor cost data can be found. The estimates 
are performed at the lowest level of the work breakdown 
Structure. 
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Or wprior relatable efforts, the copy of the 
previous cost report is attached if the estimated hours 
bid are identical oor directly related in terms of 
eq~~eivalent techtical complexity. The proposal cost 
analyst will review the incurred labor hour costs on 
sample task work orders to audit the validity and 
accuracy of the prior related task. 

The proposal analyst must search for more detailed 
evidence if the prior relatable effort is of varying 
Comerexity to the present effort. The labor hours 
currently estimated are compared directly to a similar 
Pmmwert for which verification of incurred hours emist. 
The proposal analyst compares the present effort to the 
prior related task by developing a ratio based on the 
relative technical complexities of the two tasks. The 
percentage difference must be explained by identifiable 
documentary evidence such as the following: 


1. Differences in number of units to be assembled. 
2. Variance in assembly component count. 

3. Differences in number and type of cables. 

4. Differences in testing requirements. 

5. Comparison of technical and performance 


differences. 
6. Size or weight differences. 


TO aemreve consistency on the Direct Labor 
Narrative Report, the prior effort is assigned a 
complexity base value of one. The narrative portion 


should specifically cite specification paragraphs which 
account for the difference in the plus or minus 
technical requirement. 

If no background verifiable data can be found, the 
narrative should explain how the direct labor costs are 
derived. Sources such as conceptual estimating guides 
Should be indicated by title; “for example, "Electronic 
Gectebetimating Data", by Fred Hartmeyer. The basis 
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for level of effort proposal estimating should also be 
stated. The reasons or rationale for labor cost 
estimates which are purely judgmental is also 
documented by the proposal analyst. 

The cost accounting system for Litton, incompomated 
in the computerized on-line system, provides 
historical data at the work order level, task level, 
and project level by expense center code, for direct 
labor costs. An annual labor escalation factor 
consistent with that utilized in direct labor rate 
projections is applied to estimates based on prior 
related efforts. 

The historical data utilized for direct labor 
estimating includes the following: 

1. Direct Labor Hour Audit Report: shows 
manufacturing work order closures for the latest 
two years by assembly number. A ratio for unit 
average hours by expense center code for each 
part number is derived by summation of the 
manufacturing work order closure hours. 

2. Proposal Direct Hours Report: shows two years of 
labor hours data for all subassemblies in the 
requested assembly. This report is utilized for 
manufacturing related tasks. 

3. Material Work Order History Cost Estimating 
Summary Report: summarizes total unit hours by 
expense center code for the requested assembly 
which Litton will locally manufacture. 

4. Contract Cost Status" Report: “provides, for ten 
manufacturing tasks, the historical data base 
for similar tasks. 

Litton Applied Technology’s methodology for direct 
labor estimates for development programs is similar to 
the process described earlier for all labor hours 


estimates. The only added feature for development 
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program estimates is an increased emphasis’ on 
documentation via narrative analysis for complexity 
adjustments. The proposal analyst applies appropriate 
@irect labor hour bid rates. The proposed bid rates 
are determined by examining the period of performance 
of the program plan and arriving at the midpoint of the 
effort. Historical bid rates are determined by 
dividing the quarterly total dollars expended by the 
quarterly total labor hours incurred for each expense 
center code. 

The other direct costs included in every Litton 
development or production program estimate are: 

ilies. Vendor Nonrecurring and Tooling: These represent 
the vendor costs associated with development, 
start-up, and tooling costs to produce and 
deliver equipment. The basis for this estimate 
consists of written vendor quotations or 
hietwordeal data such as wrier purchase orders or 
project cost reports. 

as Travel: Travel estimates consist of 
Pinca Petmuaighon ~ndesubsistence,costsadirectly 
associated with the program estimate. 

3. Field Service Differential: These costs include 
additional compensation over and above base 
salary expenses and per diem, which serve as an 
incentive to field support personnel on 
assignment. 

i «Semyviece Cembersiae The thmace service centers are 
Reprographic Services, Programming Services, and 
Word Processing Services. The cost estimates use 
hourly billing rates based on forecasted 
wtilizatien amd operatieomal costs for thesservice 
center. The costs are estimated either by prior 
verifiable experience or the estimated number of 


hours the service will take. 
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Litton Applied Technology also uses estimates of 
direct labor hours in their derivation of imderect pool 
costs. The six indirect pools are: 

1. Material overhead: This rate is determined by 
the ratio of annual total indirect costs for each 
expense center code divided by the total dollar 
value of the material base, including scrap and 
raw stock. 

2. Fringe benefits: This rate is calculated by the 
ratio of annual fringe benefit expense costs 
divided by the total division labor dollars 


expended. 
3. Sunnyvale plant overhead and Georgia operations 
overhead: Both pool rates are based on total 


plant overhead dollars expended annually divided 
by total division direct labor dollars plus their 
associated fringe benefit costs. 

4. Field service overhead: This rate is based on 
total annual overhead costs divided by direct 
labor costs. 

5. General and administrative expenses: These 
expenses consist of independent research and 
development, bid and proposal cost estimating, 
executive staff, general accounting, resource 
allocation and control, and defense systems 
business development costs. The rate is based on 
total expenses divided by the forecasted total 
costs sum allocable to contracts. 

Besides the indirect rates, direct factors such as 
manufacturing overtime premium, material raw stock, and 
manufacturing and engineering support services are also 
allocated. Engineering support functions include 
software engineering and development, and engineering 
design support. Manufacturing support services include 
operations comerol ; shipping, and the test @direeue ace 
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Samet . Mhe rate for these functions is based on the 
ratio of the six month historical average of direct 
labor hours for each service. 


me ACTORAL COSTS “FOR THE AN/ALR-67 RADAR WARNING 

REGET VER 

Liw@ron Applied hechmologyy “received Shes first 
Cemtract in 1975 umder a cost plus fixed fee basis from 
COMNAVAIRSYSCOM, Washington, Dee. , =o, ce velown the 
AN/ALR-67 Radar Warning Receiver. Due to budget 
G@nstraints at the "Navy level, this initial concept 
exploration contract was cancelled three times between 
1975 and 1979. During the interim periods Litton 
Boplied lechmolreay utilized their own funds to continue 
the development with a skeleton force of five to ten 
people. Litton estimates they spent $400,000 of their 
own funds during 1976 on the AN/ALR-67 development, 
with the hope of being reimbursed. The initial 
eontract amount was On $680,000. Contract 
modifications increased this amount tO wo), 0007, 000 
before funds were temporarily shut down in 1976. 
Subsequent modifications, eventually totalling 100 
mlverpetnher, increased the final development cost total 
to $6,530,000 under the contracted amount. Liveton 
Applied Technology actually spent oro 541,541, 
Anewrring® a, $4,000,000 cost overrun. Ninety-four 
percent of the development effort occurred between 1975 
ao eg 7S, ea th the remainder of the effort continuing 
through 1980. A total of seven prototype models were 
built between 1975 and 1980. 

The next contract covered the time frame from 
October 1982 to December 19805. ipess contract called 
for limited production of 43 ALR-67 systems. PO ens 
production start-up endeavor a fixed price incentive 


type contract was negotiated, placing more of the cost 
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risk on Litton Applied “~Gechnolopgy) = tiemcel len price 
on this contract wa $46,600,000 for the 43 systems. 

Since 1985, a full scale production contract has 
been awarded by the Navy for 200 additional systems at 
a total price of $103,000,000. This contract is still 
ongoing. 


Table 4 presents a budget and actual expenditure 


summary for contract N00019-75-C-0390. The actual 
development costs were broken into: 

Software design $ 1,300,000 

Hardware design $ 5,100,000 

Hardware fabrication $ 2,200,000 

Data $ 1,100,000 

Test requirements $ 800,000 

TOTAL $10,500,000 


The actual costs for the limited production 
contract were. 


Produc tilondivzamn $11,400,000 
Tooling and tests $ 5,400,000 
Data $ 3,000,000 
Hardware ~ $28,600,000 
TOTAL $48,400,000 


G. PREVIEW OF NEXT CHAPTER 

Chapter Five analyzes the cost data through a Drief 
variance analysis of the listed cost elements. It also 
includes a summary of the findings from the case study 
interviews, as well as an examination of possible new 
directions and trends for cost estimation of SOA 
Dro,ecrs. 
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V. CASE ANALYSIS AND FUTURE TRENDS 


A. INTRODUCTION 


This chapter provides an analysis of the cost 
estimating methodology used by Litton Applied 
Technology. Emerging trends in measuring and 
controlling SOA costs for the future are discussed. 
Specifically, this chapter provides the following: 

1. A variance analysis of the AN/ALR-67 Radar 
Warning Receiver development program. 

2. An overall analysis of Litton Applied 
Technology’s cost estimating system. 


3. A discussion of current concerns and directions 
within the cost estimating field, based on case 
Study. 

4. Thoughts about the roles cost estimators must 


assume dniSeiewrutu;te. 

Br. Comparison of Litton’s cost estimating process to 
the current direction of the cost estimating 
field. 


B. VARIANCE ANALYSIS OF THE DEVELOPMENT PROGRAM OF THE 
AN/ALR-67 RADAR WARNING RECEIVER 
The majority of cost ovérruns for) the curren Tee 
cycle of the AN/ALR-67 Radar Warning Receiver occurred 


in the conceptual exploration period between 1975 and 


1980. Variance analysis of the development costs 
indicates a cost overrun of $4,011,541, or 61.4 
perce The second largest contributor to the total 


cost variance was engineering services and systems 


software support, which overran the funded amount by 
5.8 percent, or $206,665. Schedule variances could not 
be calculated from the data provided, since ecie 


category for funded costs was not further subdivided 


36 


into budgeted cost of work scheduled and budgeted cost 
of work performed. 

PMeQtimetscusasionsewith company officials [|Ref. 27], 
the primary reasons for the cost variances in the 
development phase were: 

1. The sporadic nature of government funding. 

During the years 1975-1979, the project was 
stopped three times by the Navy due to funding 
constraints. The miscellaneous work stoppage and 
startup costs associated with this type of 
uncertainty were not completely compensated for 
by the Navy. 

2. Poor software cost estimating. The AN/ALR-67 was 
the first Litton Applied Technology system to 
incorporate extensive software technology. 
Several cost analysts stated the amount of labor 
hours estimated to write the lines of code for 
the software development programs were 
Significantly underestimated. There were no 
analogous programs to refer to for cost history 
comparisons. Litton Applied Technology does not 
utilize any generic cost estimation model in 
software or hardware. 

oo Inadequate definition of the work breakdown 
structure. During the years 1975-1980, the work 
breakdown structure was developed by painstaking 
manual methods, rather than use of the current 
on-line drafting capability. Litton engineers 
found it difficult during the initial development 
phase to segregate the AN/ALR-67 prototype models 
into clearly defined lower level elements. This 
lack of definition led to problems of 
underallocation of funds as actual costs began 
re) ate oer 
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Inflation and its effects on material costs. The 
years 1975-1980 were periods of rapid price 
increase for many defense electronics materials. 
As delays due to work stoppage mounted in these 
initial years, original material estimates became 
outdated. As a result, material prices were 
notably higher when purchased at the point of 
usage. 

Effect of experience. The various work stoppages 
during the development phase hindered the 
learning curve progression to an unmeasurable 
degree. As a result, more rework than originally 
planned occurred, especially in software 
development. 

Numerous contract modifications. Litton budget 
analysts estimated that 300 modifications due to 
engineering change proposals were added to the 
AN/ALR-67 development. These modifications 
accounted for the funded increase to $6,500,000 
from the original contract amount of $680,000. 
Most modifications involved increased integration 
of circuits to provide added performance within 
the same size constraints. 

Schedule pressure. During 1978, considerable 
pressure was exerted by the Navy to accelerate 
fabrication of the engineering development 
models. This pressure led to an unplanned 
increase in the number of workers and an increase 
in rework, both of which contributed to cost 
increases. 

Lack of formal controls. Litton Applied 
Technology did not have a well-documented control 
system of checklists and unplanned audits in the 
period 1975-1980. Internal investigations of 
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variance were not available for review for 
contracts during this period. This era also 
predated the DOD procurement reform measures of 
the mid-1980’s, which required more stringent 


control systems by government contractors. 


C. OVERALL ANALYSIS OF LITTON APPLIED TECHNOLOGY’S 
COST ESTIMATING SYSTEM 
This analysis of the cost estimating system for the 
AN/ALR-67 Radar Warning Receiver and Litton Applied 
Technology is based on the six managerial subsystem 
characteristics of the Katz and Rosenzweig Model, which 


are environmental, Tecan ica lk, goals and values, 
psychological, managerial, and structural. [Ref. 27] 
1. Environmental 


The environment which influences Litton Applied 
Technology is much different in 1987 than it was during 
IS 6. ; te end of the development period® tne Iso 
Applied Technology was still a separate company from 
Pr tt Ones so it could not depend on large corporate 
Resources for assistance. Applied Technology 
encountered erratic government funding for its first 


major weapons system to use software extensively. 


ALSO , competition from other defense electronics 
companies was not as intense in 1980 as in 1987. For 
example, the concepts of contractor teaming, dual 


sourcing, and leader-follower for development projects 
were not introduced to the defense industry until 
several years after the conceptual exploration phase 
for the AN/ALR-67 was’ completed. These external 
environmental factors which differentiate the period 
1975-1980 from 1987 allow insight into how Applied 
Technology has evolved to meet the current defense 


climate via structural and technical changes. Defense 
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contractors, in 1987, face (mere Sen tence msc nu Ginny won 
cost performance than they encountered in 1980 for 
development contracts. 
2. ltecmrlcall 

Litton Applied Technology has greatly improved 
its cost estimating capabilities. They currently use a 
new IBM mainframe computer system to keep track of 
price histories, work breakdown structures, and vendor 
quotes. However, they have made a conscious decision 
not to utilize generic parametric cost estimating 
models which could be adapted to the mainframe system. 
Litton cost analysts believe the physical dimensions of 
their radar warning receivers are too small for 
accurate application of the RCA Price cost estimating 
model, for example. As a result, all cost improvements 
are geared toward upgrading their bottoms-up cost 
estimating capability. Their emphasis is on cost 
performance trend analysis for their own products. 

Litton Applied Technology’s cost estimators 
understand the difficulties of accurate estimation for 
the new integrated circuits of the future. The SOA for 
Litton’s radar warning receivers will extend into 
increasing miniaturization of components, which will be 
more reliable and capable than their predecessors. 
Litton estimators currently expect vendor-purchased 
electronic components to decrease in price in the next 
few years due to better integrated circuit technology 
and yield improvement. Litton will continue to rely on 
its mainframe cost collection system and group judgment 
techniques to estimate costs for SOA extensions, with 
no foreseeable plans to incorporate regression analysis 
or cost prediction models. Finally, due to its 
mainframe capabilities, Litton rarely uses personal 


computers in its cost estimation process. 


90 


a, Struceure 


Structurally, Ee AeeO mM Applied Technology 
utilizes a matrix organization concept for its cost 
estimation process. With this system they draft 


members from functional departments to participate in 
the cost proposal process, terminating the team upon 
completion of the negotiated contract. 

Litton Applied Technology has’ the functional 
department heads review and approve the proposal team’s 
recommendations. This current system does not allow 
any one individual the Opportunity to wield an 
overwhelming influence on cost estimates. The 
situation differed slightly from 1975-1980, when the 
AN/ALR-67 program management team remained intact and 
exerted considerable independence. 

4. Psychosocial 

At the psychosocial level, Litton Applied 
Technology has made tremendous strides for its 
employees. At its south San Jose location, a new 
building with plush offices, a complete cafeteria, and 


recreational facilities have been added to improve 


company morale. Litton requires all employees, 
including executives, to refer to each other by first 
names only. Litton’s executives also meet annually at 


special retreats’ to discuss the current state of the 
company and its future strategy. 

Most of Litton’s cost estimators are relatively 
young, with an accounting background from college. 
Litton trains these estimators themselves, rather than 
relying on outside cost estimating seminars. They do 
not participate in the Space Systems Cost Analysis 
Group or belong to the [International Society of 
Parametric Analysts. Litton has recently published a 
comprehensive "Pricing and Estimation Manual" for its 
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cost analysts. However, in 1 9siG7 few written 
guidelines were available for use by the cost analysts. 
5. Managerial 

The managerial process at Litton Applied 
Technology is heavily weighted toward planning and 
organization. Almost every area of the building 
carries its own blackboard and space for meeting rooms. 
According to the Pricing and Estimation Manual, most 
meetings have set time limits of five to ten minutes 


per topic. Litton exhibits a strong vertical decision- 


making structure, since functional department levels 
usually do not deal with other functional lower levels 
unless prior liaison has been arranged by the 


department heads. 
6. Goals and Values 

Litton Applied Technology’s goals and values 
emphasize quality workmanship and reliability. Their 
marketing personnel’s primary emphasis is on the 
excellent performance and versatility of the radar 
warning receivers in combat situations. At proposal 
team meetings, the actual cost estimates are usually 
not the primary issue. Proposal meeting agendas 
concentrate on analysis of he competition and 
packaging and presentation of the proposal itself. 

To Control Mechanisms 

The AN/ALR-67’s costs are currently controlled 
primarily through vendor control and monitoring of 
labor hours. Litton Applied Technology has established 
a vendor qualification program to meet its raw material 
needs, in order to minimize material defects. Vendor 
costs are controlled by trend comparison with prior 
related efforts on the mainframe history files. Litton 
Applied Technology’s internal auditors monitor, via 
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surprise visits, the recording of labor hours by each 
cost element. 

In this researcher’s opinion, Litton Applied 
Technology could more effectively utilize postaudits as 
a control mechanism. Their cost analysts receive 
little feedback on proposal cost estimates which are 
rejected in favor of another contractor. Postaudit 
conferences with cost analysis can provide an effective 
"lessons learned" benefit for the future. 


D. CURRENT CONCERNS AND TRENDS IN COST ESTIMATION 

The current focus of practitioners of SOA cost 
estimating is on increasing the range and depth of the 
data base. During an interview with Dr. E.N. Dodson on 
16 October 1987, he indicated a primary problem in cost 
estimating is the lack of good historical data to 


substantiate cost prediction models. Dodson indicated 
most data for cost models comes’ from after-the-fact 
analysis of costs at the production level. Dodson 


feels more rigorous analysis of costs at the design 
Stage is critical for accounting for technological 
change. To properly understand the cost impact of new 
technology, Dodson stated cost estimators must become 
more knowledgeable in engineering. En@ineering 
backgrounds would enable estimators to better locate 
the cost drivers which are influenced by technology 
parameters. 

Another concern of modern cost estimators, 
accerdings to Dr. Dodson, is the inability to influence 
potential costs during the initial phases of the design 
process. Dodson feels cost estimators should develop 
the capability to review performance parameters in the 
design process, and subsequently advise design 


engineers as to the legitimacy of the specifications. 
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For example, cost analysts should be capable of asking 
if less rigorous specifications could be substituted 
for MNovrerwr ll porous ones. 


One shortcoming of many cost estimators, according 


to Dr. Dodson, is their failure to recognize old 
technology under the guise of new technology. Once 
estimators increase their technical knowledge, this 
mistake will be less likely to occur. Also, better 
subdivision of the work breakdown structure into 


definable cost elements should sort out old and new 
technology. 
Dodson feels his theories on the use of surface 


fitting techniques and regression analysis to estimate 


SOA extension costs are too time-consuming and 
expensive. Most corporations involved in government 
cost proposals could not efficiently utilize these 
techniques. As Dodson indicates, DOD agencies prefer 


to audit detailed systems engineered estimates which 
substantiate every cost element. DOD agencies do not 
require parametric estimates of new developments, so 
cost estimators currently use cost estimation models 
only as a checking mechanism. 

For SOA extension measurement, Dodson states his 
current focus is on transforming performance parameters 
to design parameters, and ultimately to cost. He 
emphasizes efficiency parameters should be utilized in 
selecting key technological variables. For example, he 
suggests receiver sensitivity, receiver bandwidth, and 
receiver frequency might be good indicators of SOA 
advance for radar systems. 

Finally, Dr. Dodson states risk analysis of cost 
estimates is another area which requires further study. 
This area should quantify the probabilities that 
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Mmaerruns or overruns Swill occur in development 
projects. 

Stan Swales, the leading cost estimator for GTE 
Government Products and a member of the Space Systems 
Cost Analysis Group, also shared his opinion on the 
current trends of the cost estimating field. [Ref. 28] 
Mr. Swales recommends the use of Expert Systems, a 
branch of artificial intelligence that is concerned 
with emulating the problem solving processes of human 
experts. On 28 July 1987 he stated "Expert Systems is 
the wave of the future." He feels Expert Systems 
technology optimizes the exchange of information and 
improves the evaluation of data. For Mr. Swales’ 
sys tem, cost engineers interview experts and program 
their answers into the knowledge base of the Expert 
osc CGM. The knowledge base should eventually consist 
of rules, cost estimating relationships, and a numeric 
data base. The next phase of Swales’ Expert System 
concerns the inference engine, which contains’ the 
Sentrols strategies and control structures for the 
model. The cost predictor portion of the model 
develops cost estimates for programs by combining the 
control given by the inference engine with the user’s 
input data and the knowledge base. Although Swales 
utilizes this model to predict costs of programs in the 
conceptual empl oration stage, it has not been 
completely implemented into GTE’s formal cost 
estimating system. The current application of his 
PespeGbE System does not specifically quantify SOA 
extension costs. 


Stan Swales states Expert Systems solve cost 
estimating problems by heuristic, or rules of thumb 
methods. He believes Expert Systems, with their non- 
algorithmic solutions, can more closely mimic 
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judgmental cost estimating process than conventional 
methods can. In his presentation on Expert Systems to 
the Space Systems Cost Analysis Group on 24 September 
1986, he drew three conclusions: 

1. Expert Systems rapidly adapt to any type program, 
with benefits increasing exponentially with 
program complexity. 

2. Expert Systems cost models designed for desk-top 
computers reduce proposal times and improve 
program cost control. 

ws Expert Systems techniques reduce potential for 
program cost overruns. 


E. EMERGING ROLES COST ESTIMATORS MUST ASSUME IN THE 

FUTURE 

This researcher discovered through personal 
interviews with Dr. E.N. Dodson and Stan Swales that 
most leading cost estimators have also attained a broad 
engineering background. During a presentation to the 
International Society of Parametric Analysts during 
March 1987, Mr. Edward Laughlin discussed the role cost 
estimators must assume _ to properly assess future 
technologies. [Ref. 29] 


Laughlin stated cost estimators must improve their 


understanding of the total implications of new 
technology. He believes analysts must develop closer 
coordination with the engineering community to better 
understand the technical implications of resource 
management decisions. They must be fluent in 
acquisition, budgetary, and technical languages since 
the cost analyst will become an integrator of staff 
bnformation in the = =hiteare. For example, Laughlin 
Wie ie se 


We can’t afford _to look blank when people start 
talking about flexible manufacturtiua {f tper soperece 
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robotics, and composite materials because we’re gpoin 
to be tasked to recommend important and seas 
y's 


decisions hat Gam’ t.—b d 
knowledge. [Ref. 304 e made with yesterda 


Laughlin believes cost estimators must have 
diversified experience and a broad based education to 
understand the cost impact of new technology. He 
recommends that data bases concentrate on accumulating 
information on technological cost drivers, rather than 
massive storage of cost histories. He also recommends 
that cost estimators continually update their education 
and training on future hardware and software trends, 
such as computer aided design (CAD) and artificial 
intelligence. Finally, Laughlin recommends engineering 
specialists become permanent members’ of cost analysis 
divisions to increase _ the interdependency between 
engineers and cost estimators. 


F. COMPARISON OF LITTON’S COST ESTIMATING PROGRAM TO 

CURRENT DIRECTION OF THE ESTIMATING FIELD 

Litton Applied Technology’s cost estimating system 
has evolved into a bottoms-up system engineered 
procedure which makes extensive use of a deep data base 
of prior related costs. Their data base consists of 
price histories for parts which date back fifteen 
years. Extensive labor rate data for carefully defined 
job descriptions are also entered into the data base. 
Their ability to adapt to the future direction of the 
cost estimating field, and their capability to estimate 
the cost of SOA advances, are assessed in the following 
paragraphs. 

1. SOA Advance Measurements 

Litton does not specifically measure the cost 

of the SOA extension. Dodson’s use of ellipsoid 
surfaces to define the SOA performance parameters has 


not been addressed by Litton Applied Technology. Other 
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SOA measurement techniques, such as Alexander’s 
regression analysis of primary parameters, are also not 
uti Dived Even for conceptual exploration phase 
projects, Litton relies on consistent application of 
its bottoms-up system. The proposal analyst and the 
functional engineers use analogy methods when exact 
data from the WBS elements is’ not available. Litton 
cost estimators do not use generic parametric cost 
models such as PRICE or FAST. 

This researecien, feels Litton’s expensive 
commitment to their bottoms-up method for SOA extension 
projects does not provide their analysts sufficient 
flexibility for "what-if" analysis. Their exploration 
of different alternatives is limited, since detailed 
documentation efforts at the WBS lower levels are 
mandated for each acquisition phase. Their exclusive 
use of the bottoms-up method does not’ provide 
information on the expense involved in increasing SOA 
parameters, such as receiver sensitivity, ina timely 
manner. 

2. Hxpertisys@ens 

Artificial intelligence applications to cost 
estimating will become more prevalent in the future. 
Litton Applied Technology, with its extensive cost data 
base, is missing an opportunity to improve its 
estimating capability by not investigating the benefits 
of this field. They already possess extensive computer 
capabilities which, coupled with their knowledge of 
radar warning receivers, makes them ideal candidates 
for expert systems. 

Some of the opportunities the use of expert 
systems would present to Litton are listed. 
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a. Learnime Tool 


Expert systems give users the capability to 
obtain the experts’ consensus judgment on what costs 


should be, based on the users’ answers to system 
queries. Litton’s cost analysts do not have 
postgraduate education Or extensive engineering 


backgrounds, but implementation of an expert system 
would rapidly increase their capabilities to mimic an 
expert’s answer. 
b. Warehouse for Cost Estimating Knowledge 
The expert system’s knowledge data base 
would incorporate the expertise of Litton’s' most 
capable professionals. Their proposal analysis teams 
would gain consistency through use of the expert 
system, since recommendations by the system would be 
based on the same heuristic rules input to the system. 
Ce. Pre-planning Tool 
Expert systems, normally designed for 
personal computers, provide the rapid flexibility of 
what-if analysis missing in Litton Applied Technology’s 
euriTent SVS ten. 
d. Research Tool 
Bitven*s cost estimators could perform 
research on new projects by studying the implications 
of different parameters in a quick, iterative fashion. 
Expert systems are expensive, and most are 
still in formative stages. This researcher recommends 
ErVtton track the progress of artificial intelligence in 
the cost estimating field. Cost benefit analysis 
should be performed to see if their data based 
management system would be significantly improved. 
ons Influence of Costs in the Design Stage 
Cost estimators at Litton do not yet impact 


planning of performance parameters at the design stage. 
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In the proposal cycle the cost analyst collects inputs 
from the WBS elements, after the specifications are 
already decided. However, with the rapid pace of 
technology, Litton will gain more effective control of 
costs through more rigorous’ analysis of costs earlier 
in the design stage. 


4. Understanding of Total Implications of New 


Technology 

Most Litton cost analysts have accounting 
backgrounds. They do not regularly attend professional 
cost estimating seminars. For Litton to improve its 


ability to estimate SOA extension costs, the estimators 
must understand how new technology will influence 
costs. The cost analysts should receive training on 
fields such as artificial intelligence and computer 
aided design. This researcher recommends high 
technology firms like Litton should consider inclusion 
of engineers Knowledgeable in future hardware trends in 
the cost estimating division. 
oO". Isolation of Cost Drivers 
Litton’s ability to estimate SOA extension 
costs and control development costs would be enhanced 
by a thorough analysis of which design and performance 
parameters generate the most significant influence on 
cost. Isolation of the key cost drivers for radar 
warning receivers would enable planners to concentrate 
on cost reductions in these areas. 


F. SUMMARY 

This chapter’s primary goal has been to analyze key 
features of Litton Applied Technology through specific 
study of the AN/ALR-67’s development program. 
Preceding sections provided insight into the direction 
of the cost estimating field, and the qualities cost 
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analysts must develop in the future, before reliable 
cost estimates for SOA extensions can be made. 
Mme next chapter provides the final summary and 


recommendations for future study. 
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VI. SUMMARY AND CONCLUSIONS 


A. @INTRODUC ETON 

This chapter presents the summary and conclusions 
for this thesis. The primary focus of the research 
has been to determine how deéetense (industry scoce 
estimators measure the costs of SOA extensions for new 
weapons systems. The researcher utilized a case study, 
of Litton Applied Technology’s development of the 
AN/ALR-67 Radar Warning Receiver to present an actual 
example. 

The literature review section highlighted the work 
of Dr.  E.N. Dodsonmmot General Research Corporation, a 
pioneer in the field of SOA measurement. Cost 
prediction models by selected authors were also 
examined to demonstrate how cost estimation theory is 
transformed to actual data. The last chapter 
highlighted current concerns and future trends in the 
cost estimating field. 


B. LITERATURE REVIEW 

The literature review section began with Ostwald’s 
G@efinition of cost estimating relationships as 
functional models that mathematically describe the 
costs of components as functions of one or more 
independent variables. Ostwald also described the 
roles cost estimators play in technological forecasting 
and development of cost indices. 

Dr. E.N. Dodson has established himself as one of 
the foremost practitioners of quantitative measurement 
of SOA advances. He described the SOA for a particular 
system as an n-dimensional function at a particular 
point in time, with n representing the number of SOA 


design characterd sre. He indicavet peome tric 
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Surfaces could illustrate the tradeoff between design 
characteristics for SOA advances. Dodson specifically 
developed SOA equations for solid propellant missiles. 
He also constructed SOA indices by geometric measures 
of data characteristics on an ellipsoid surface. 


Alexander and Nelson of Rand Corporation measured 


teennolopical change through multiple regression 
analysis. their studiesmindicated performance, rather 
than technical, parameters were better descriptors of 


SOA advance. 

Hovanssian recommended electronic systems include 
Customer acceptance parameters at the development 
stage, such as operator approval, life cycle cost, and 
amount of maintenance required per operating hour. 
Cooley developed the TRAM model, which could analyze 
the effects of budget reductions on SOA development 
projects. 

Other studies of technological advance were 
conducted by Gordon and Munson, who used experts to 
choose the proper parameters for their SOA extension 
eam@iations. Hhinally, Marino further refined the role of 
surface analysis in SOA measurements. 


C. COST MODELS 
Four different cost estimating models have been 
examined in earlier chapters. The two primary 
inferedicnts: in their devebopment were: 
1. Derivation of the work breakdown structure. 
2. Development of cost estimating relationships for 
eaeh cest element. 
The cost estimating relationships at the lowest 
level of the WBS were derived by a variety of methods 
for these models, from parametric analysis to analogy 


comparisons. The cost models all provided a hierarchy 
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of estimating systems, depending on the data available 
to the analyst. In Dodson’s models, he recommended 
regression equations should only be used as a last 
resort. 

ine FAST model utilizes technological 
characteristics such as energy outputs for its 
foundation. Among the products of the FAST model are 
projections of funding requirements for each stage of 
the development cycle. 


D. LITTON’S COST ESTIMATION SYSTEM 

Litton Applied Technology utilizes a detailed 
bottoms-up cost estimation approach for all electronics 
systems projects, including those at the early 
development stage. They do not use parametric cost 
estimation models; they rely on an elaborate component 
cost history data dating back to 1975stored on their 
IBM Mainframe computer. Their cost estimation process 
involves eliciting judgments from many personnel from 
different functions organized in a matrix team 
structure. The final cost estimate decision is arrived 
at only after a series of judgmental steps at different 
committee meetings. 

Litton Applied Technology experienced a_ecost 
overrun of 61 percent for the development phase of the 
AN/ALR-67 Radar Warning Receiver. Some of the reasons 
explaining this cost growth were Litton Applied 
Technology’s lack of familiarity with software cost 
estimation, numerous additions of modifications, 
intermittent work stoppages, and poor definition of 
manually drafted work breakdown structures. They did 
not use any specific techniques or parametric cost 
models to estimate the actual cost of the SOA 
extension. 
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E. CURRENT DIRECTIONS 

Multiple regression analysis, trade off surface 
analysis, and factor analysis have been used by various 
practitioners to measure and estimate the actual costs 
of SOA extensions. Research has now focused on 
analysis of the efficiency characteristics of the 
primary parameters which most completely describe the 
technological advancement. Interviews with Dr. Dodson 
Suggest the results of these SOA extension studies have 
not been integrated into the current cost estimation 
models. 

Other cost estimators, like Stan Swales, predict 
artificial intelligence will be used more extensively 
in cost estimation models in the future. However, 
there are no indications Expert Systems methodology has 
specifically addressed the issue of SOA extension 


measurement in actual weapons systems applications. 


poe SUlURE ROLES 

The cost estimator of the future must assume the 
role of an integrator between engineering and cost 
analysis. With many electronic hardware costs actually 
decreasing exponentially due to technological 
innovation, cost estimators must continually update 
their knowledge on the latest technological 
innovations. Data bases should contain information on 
the cost, performance, and physical characteristics of 
the primary cost drivers which influence technological 
advancement. 

DOD project offices can reduce overruns’ in SOA 
development programs by decreasing the amount of 
administrative and specification changes during the 


conceptual exploration stage. However, to facilitate 
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this, government cost estimators must become involved 
early in the design process, and advise engineers as to 
the cost tradeoffs of specification changes. 


G. CONCLUSIONS 
The following conclusions can be drawn from this 
study: 

1. The systems engineered cost estimating approach 
is relied upon by Litton Applied Technology for 
its cost estimation methodology. 

2. Various methods, such as Dr. Dodson’s regression 
analysis and surface fitting techniques, have 
succeeded in measuring the cost of SOA 
extensions. These methods have not been utilized 
in actual applications by industry or DOD in 
their cost estimating models. 

3. The fundamental ingredient to parametric analysis 
of technological advancements is specification of 
the relationship between cost, performance, and 
physical characteristics of the primary cost 
drivers. 

4. The development program cost estimate requires a 
detailed work breakdown structure ina bottoms-up 
method to prevent cost omissions. Errors in this 
area are a frequent cause of cost overruns. 

i. Cost estimators must possess technical knowledge 
of the task being estimated if they are going to 
competently measure the costs of technological 
advancement. 

6. The cost estimating field is dynamically 
increasing its knowledge in the area of 
parametric desk-top models and expert systems. 
These systems are useful for rapid analysis of 
different possibilities. 
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H. RECOMMENDATIONS FOR FUTURE STUDY 

Future thesis efforts are needed to track the 
progress of the cost estimating profession in its 
endeavor to quantify the costs of SOA extensions. 
Also, future studies should review the feasibility of a 
Mapee, Universal data Dase for all electronics systems, 
from which defense contractors and DOD cost estimators 
Scola extract information. Pooled efforts might 
accelerate the progress in transforming technological 


advancement theory to actual practice. 
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